[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:03 dist-test-slave-vxj2 krb5kdc[6018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:03 dist-test-slave-vxj2 krb5kdc[6018](info): set up 2 sockets
Aug 09 04:40:03 dist-test-slave-vxj2 krb5kdc[6018](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 04:40:04.650974 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:04 dist-test-slave-vxj2 krb5kdc[6018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:40:04.784834 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:42231
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:42231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:04.919524 6033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:04.919863 6033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:04.919945 6033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:04.924023 6033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:04.924114 6033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:04.924160 6033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:04.924219 6033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:04.924264 6033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:04.929811 6033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:42231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:42231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:04.931056 6033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:04.932224 6033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:04.942147 6040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:04.942167 6043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:04.942220 6033 server_base.cc:1034] running on GCE node
W20260809 04:40:04.942229 6041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:04.943068 6033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:04.944443 6033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:04.945634 6033 hybrid_clock.cc:648] HybridClock initialized: now 1786250404945616 us; error 39 us; skew 500 ppm
Aug 09 04:40:04 dist-test-slave-vxj2 krb5kdc[6018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250404, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:04.948588 6033 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:40:04.949668 6033 webserver.cc:460] Webserver started at http://127.5.222.254:33107/ using document root <none> and password file <none>
I20260809 04:40:04.950309 6033 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:04.950402 6033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:04.950723 6033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:04.952906 6033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "d07f4b51143e47a4bbc0269a69b43cd6"
format_stamp: "Formatted at 2026-08-09 04:40:04 on dist-test-slave-vxj2"
server_key: "22206dd6f3ba1178ee811c4b7b71d38e"
server_key_iv: "18680a35b500263a1e357b9d0e6d660c"
server_key_version: "encryptionkey@0"
I20260809 04:40:04.953665 6033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "d07f4b51143e47a4bbc0269a69b43cd6"
format_stamp: "Formatted at 2026-08-09 04:40:04 on dist-test-slave-vxj2"
server_key: "22206dd6f3ba1178ee811c4b7b71d38e"
server_key_iv: "18680a35b500263a1e357b9d0e6d660c"
server_key_version: "encryptionkey@0"
I20260809 04:40:04.957803 6033 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:40:04.960264 6052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:04.961645 6033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:04.961784 6033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "d07f4b51143e47a4bbc0269a69b43cd6"
format_stamp: "Formatted at 2026-08-09 04:40:04 on dist-test-slave-vxj2"
server_key: "22206dd6f3ba1178ee811c4b7b71d38e"
server_key_iv: "18680a35b500263a1e357b9d0e6d660c"
server_key_version: "encryptionkey@0"
I20260809 04:40:04.961905 6033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:04.980398 6033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:04.983347 6033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:04.983553 6033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:04.991411 6033 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:42231
I20260809 04:40:04.991441 6111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:42231 every 8 connection(s)
I20260809 04:40:04.992395 6033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:04.995366 6112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:05.001243 6112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6: Bootstrap starting.
I20260809 04:40:05.002524 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 6033
I20260809 04:40:05.002713 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:05.004415 6011 external_mini_cluster.cc:1428] Setting key 080a47fcd9903b52c4ab3661515bf9a4
I20260809 04:40:05.004693 6112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:05.005733 6112 log.cc:824] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:05.008176 6112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6: No bootstrap required, opened a new log
I20260809 04:40:05.013731 6112 raft_consensus.cc:348] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07f4b51143e47a4bbc0269a69b43cd6" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 42231 } }
I20260809 04:40:05.014012 6112 raft_consensus.cc:374] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:05.014092 6112 raft_consensus.cc:729] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07f4b51143e47a4bbc0269a69b43cd6, State: Initialized, Role: FOLLOWER
I20260809 04:40:05.014616 6112 consensus_queue.cc:260] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07f4b51143e47a4bbc0269a69b43cd6" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 42231 } }
I20260809 04:40:05.014781 6112 raft_consensus.cc:388] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:05.014849 6112 raft_consensus.cc:482] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:05.014950 6112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:05.016237 6112 raft_consensus.cc:504] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07f4b51143e47a4bbc0269a69b43cd6" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 42231 } }
I20260809 04:40:05.016640 6112 leader_election.cc:302] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d07f4b51143e47a4bbc0269a69b43cd6; no voters:
I20260809 04:40:05.017016 6112 leader_election.cc:288] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:05.017123 6118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:05.017413 6118 raft_consensus.cc:686] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [term 1 LEADER]: Becoming Leader. State: Replica: d07f4b51143e47a4bbc0269a69b43cd6, State: Running, Role: LEADER
I20260809 04:40:05.017712 6118 consensus_queue.cc:237] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07f4b51143e47a4bbc0269a69b43cd6" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 42231 } }
I20260809 04:40:05.018296 6112 sys_catalog.cc:564] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:05.019040 6118 sys_catalog.cc:455] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d07f4b51143e47a4bbc0269a69b43cd6. Latest consensus state: current_term: 1 leader_uuid: "d07f4b51143e47a4bbc0269a69b43cd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07f4b51143e47a4bbc0269a69b43cd6" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 42231 } } }
I20260809 04:40:05.019150 6118 sys_catalog.cc:458] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:05.019837 6120 sys_catalog.cc:455] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d07f4b51143e47a4bbc0269a69b43cd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07f4b51143e47a4bbc0269a69b43cd6" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 42231 } } }
I20260809 04:40:05.019945 6120 sys_catalog.cc:458] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:05.020619 6124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:05.023646 6124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:05.029683 6124 catalog_manager.cc:1347] Generated new cluster ID: 828fb5f0e9384318b603bc66848e5872
I20260809 04:40:05.029744 6124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:05.034425 6116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:05.008323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38832 (local address 127.5.222.254:42231)
0809 04:40:05.008799 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:05.008821 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:05.010187 (+ 1366us) server_negotiation.cc:407] Connection header received
0809 04:40:05.013925 (+ 3738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:05.013938 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:05.014323 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:05.014617 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:05.017718 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.019365 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.021236 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.021936 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.022092 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:05.027325 (+ 5233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:05.027332 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:05.027348 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:05.027385 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:05.029571 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:05.030273 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:05.030279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:05.030285 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:05.031096 (+ 811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:05.031572 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:05.031575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:05.031577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:05.032227 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:05.032462 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:05.033051 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 04:40:05.033386 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":147,"threads_started":1}
I20260809 04:40:05.038806 6124 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:05.039670 6124 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:05.046828 6124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d07f4b51143e47a4bbc0269a69b43cd6: Generated new TSK 0
I20260809 04:40:05.047513 6124 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:40:05.080945 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:42231
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:05.198743 6139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:05.199012 6139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:05.199056 6139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:05.203190 6139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:05.203254 6139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:05.203349 6139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:05.208813 6139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:42231
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:05.209937 6139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:05.210978 6139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:05.220985 6146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:05.221019 6148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:05.220984 6145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:05.221519 6139 server_base.cc:1034] running on GCE node
I20260809 04:40:05.221921 6139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:05.222553 6139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:05.223734 6139 hybrid_clock.cc:648] HybridClock initialized: now 1786250405223723 us; error 31 us; skew 500 ppm
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250405, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:05.232081 6139 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:40:05.233081 6139 webserver.cc:460] Webserver started at http://127.5.222.193:46737/ using document root <none> and password file <none>
I20260809 04:40:05.233704 6139 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:05.233803 6139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:05.234043 6139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:05.236316 6139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "10736cc896c942819006fddd10f7d84d"
format_stamp: "Formatted at 2026-08-09 04:40:05 on dist-test-slave-vxj2"
server_key: "fca7bb919532a2c2a505cccd900a273f"
server_key_iv: "15caae173ba6b83dca5d8eb5df39491b"
server_key_version: "encryptionkey@0"
I20260809 04:40:05.236956 6139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "10736cc896c942819006fddd10f7d84d"
format_stamp: "Formatted at 2026-08-09 04:40:05 on dist-test-slave-vxj2"
server_key: "fca7bb919532a2c2a505cccd900a273f"
server_key_iv: "15caae173ba6b83dca5d8eb5df39491b"
server_key_version: "encryptionkey@0"
I20260809 04:40:05.240882 6139 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260809 04:40:05.243439 6158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:05.244638 6139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:05.244769 6139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "10736cc896c942819006fddd10f7d84d"
format_stamp: "Formatted at 2026-08-09 04:40:05 on dist-test-slave-vxj2"
server_key: "fca7bb919532a2c2a505cccd900a273f"
server_key_iv: "15caae173ba6b83dca5d8eb5df39491b"
server_key_version: "encryptionkey@0"
I20260809 04:40:05.244882 6139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:05.265515 6139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:05.268925 6139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:05.269141 6139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:05.269825 6139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:05.270696 6139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:05.270767 6139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:05.270840 6139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:05.270887 6139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:05.280893 6139 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:46097
I20260809 04:40:05.280918 6287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:46097 every 8 connection(s)
I20260809 04:40:05.281839 6139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:05.288040 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 6139
I20260809 04:40:05.288160 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:05.288393 6011 external_mini_cluster.cc:1428] Setting key d68d91bbbf1888e88f2fe6e7ba200d15
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250405, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:05.296293 6116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:05.283656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:47533 (local address 127.5.222.254:42231)
0809 04:40:05.283804 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:05.283808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:05.284609 (+ 801us) server_negotiation.cc:407] Connection header received
0809 04:40:05.286209 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:05.286212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:05.286271 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:05.286358 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:05.287506 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.288066 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.289475 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.289773 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.289861 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:05.292478 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:05.292482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:05.292485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:05.292518 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:05.294064 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:05.294691 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:05.294694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:05.294695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:05.294764 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:05.295164 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:05.295166 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:05.295167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:05.295326 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:05.295409 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:05.296078 (+ 669us) server_negotiation.cc:299] Negotiation successful
0809 04:40:05.296173 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:40:05.297570 6292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:05.284033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:42231 (local address 127.5.222.193:47533)
0809 04:40:05.284465 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:05.284507 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:05.286004 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:05.286526 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:05.286533 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:05.286877 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:05.287357 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:05.287372 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.288196 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.288201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:05.289366 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:05.289376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.289934 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.289940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:05.290103 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:05.290735 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:05.290751 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:05.292302 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:05.294178 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:05.294186 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:05.294202 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:05.294592 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:05.294857 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:05.294861 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:05.294867 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:05.295068 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:05.295449 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:05.295458 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:05.295804 (+ 346us) client_negotiation.cc:769] Sending connection context
0809 04:40:05.296052 (+ 248us) client_negotiation.cc:240] Negotiation successful
0809 04:40:05.296430 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
I20260809 04:40:05.298837 6288 heartbeater.cc:347] Connected to a master server at 127.5.222.254:42231
I20260809 04:40:05.299069 6288 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:05.299572 6288 heartbeater.cc:511] Master 127.5.222.254:42231 requested a full tablet report, sending...
I20260809 04:40:05.301353 6072 ts_manager.cc:194] Registered new tserver with Master: 10736cc896c942819006fddd10f7d84d (127.5.222.193:46097)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:40:05.302676 6072 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:47533
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:40:05.318535 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:42231
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:05.444926 6295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:05.445228 6295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:05.445302 6295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:05.449679 6295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:05.449751 6295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:05.449836 6295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:05.455356 6295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:42231
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:05.456429 6295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:05.457476 6295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:05.467227 6301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:05.467270 6302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:05.467235 6304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:05.467437 6295 server_base.cc:1034] running on GCE node
I20260809 04:40:05.467900 6295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:05.468541 6295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:05.469724 6295 hybrid_clock.cc:648] HybridClock initialized: now 1786250405469708 us; error 27 us; skew 500 ppm
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250405, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:05.472579 6295 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:40:05.473616 6295 webserver.cc:460] Webserver started at http://127.5.222.194:40293/ using document root <none> and password file <none>
I20260809 04:40:05.474148 6295 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:05.474203 6295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:05.474365 6295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:05.476516 6295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "1026fe45bee64e17bd61c3e970a74e55"
format_stamp: "Formatted at 2026-08-09 04:40:05 on dist-test-slave-vxj2"
server_key: "7a4764ef6c396b447940d5cf8d0dfa38"
server_key_iv: "1d600c3cb5236554dfbc84dfa4f63f27"
server_key_version: "encryptionkey@0"
I20260809 04:40:05.477106 6295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "1026fe45bee64e17bd61c3e970a74e55"
format_stamp: "Formatted at 2026-08-09 04:40:05 on dist-test-slave-vxj2"
server_key: "7a4764ef6c396b447940d5cf8d0dfa38"
server_key_iv: "1d600c3cb5236554dfbc84dfa4f63f27"
server_key_version: "encryptionkey@0"
I20260809 04:40:05.480986 6295 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:40:05.483547 6314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:05.484779 6295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:05.484900 6295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "1026fe45bee64e17bd61c3e970a74e55"
format_stamp: "Formatted at 2026-08-09 04:40:05 on dist-test-slave-vxj2"
server_key: "7a4764ef6c396b447940d5cf8d0dfa38"
server_key_iv: "1d600c3cb5236554dfbc84dfa4f63f27"
server_key_version: "encryptionkey@0"
I20260809 04:40:05.485046 6295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:05.508287 6295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:05.511365 6295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:05.511562 6295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:05.512166 6295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:05.513185 6295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:05.513270 6295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:05.513336 6295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:05.513376 6295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:05.524940 6295 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:40779
I20260809 04:40:05.525038 6445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:40779 every 8 connection(s)
I20260809 04:40:05.525930 6295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250405, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:05.536000 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 6295
I20260809 04:40:05.536090 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:05.536361 6011 external_mini_cluster.cc:1428] Setting key 506d4ec54613416e536affe5a727d012
I20260809 04:40:05.540565 6116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:05.527709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:57225 (local address 127.5.222.254:42231)
0809 04:40:05.527850 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:05.527854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:05.528725 (+ 871us) server_negotiation.cc:407] Connection header received
0809 04:40:05.530127 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:05.530131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:05.530194 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:05.530296 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:05.531501 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.532038 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.533380 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.533603 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.533717 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:05.536370 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:05.536373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:05.536375 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:05.536400 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:05.538180 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:05.538741 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:05.538748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:05.538753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:05.538855 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:05.539213 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:05.539220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:05.539224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:05.539502 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:05.539615 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:05.540321 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 04:40:05.540443 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:40:05.541817 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:05.528125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:42231 (local address 127.5.222.194:57225)
0809 04:40:05.528573 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:05.528614 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:05.529920 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:05.530465 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:05.530472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:05.530846 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:05.531348 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:05.531362 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.532172 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.532180 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:05.533254 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:05.533264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.533763 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.533768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:05.533954 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:05.534582 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:05.534599 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:05.536168 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:05.538296 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:05.538302 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:05.538318 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:05.538586 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:05.538972 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:05.538975 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:05.538976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:05.539122 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:05.539631 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:05.539637 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:05.540030 (+ 393us) client_negotiation.cc:769] Sending connection context
0809 04:40:05.540274 (+ 244us) client_negotiation.cc:240] Negotiation successful
0809 04:40:05.540659 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":127,"threads_started":1}
I20260809 04:40:05.543015 6446 heartbeater.cc:347] Connected to a master server at 127.5.222.254:42231
I20260809 04:40:05.543262 6446 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:05.543756 6446 heartbeater.cc:511] Master 127.5.222.254:42231 requested a full tablet report, sending...
I20260809 04:40:05.544896 6072 ts_manager.cc:194] Registered new tserver with Master: 1026fe45bee64e17bd61c3e970a74e55 (127.5.222.194:40779)
I20260809 04:40:05.545576 6072 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:57225
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:40:05.568192 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:42231
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:05.686416 6452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:05.686746 6452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:05.686838 6452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:05.692250 6452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:05.692353 6452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:05.692461 6452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:05.701419 6452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:42231
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:05.703212 6452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:05.704667 6452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:05.714423 6460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:05.714406 6459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:05.714407 6463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:05.714736 6452 server_base.cc:1034] running on GCE node
I20260809 04:40:05.715236 6452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:05.715821 6452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:05.716986 6452 hybrid_clock.cc:648] HybridClock initialized: now 1786250405716964 us; error 32 us; skew 500 ppm
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250405, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:05.720079 6452 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:40:05.720971 6452 webserver.cc:460] Webserver started at http://127.5.222.195:35363/ using document root <none> and password file <none>
I20260809 04:40:05.721561 6452 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:05.721623 6452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:05.721783 6452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:05.723852 6452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "d7772899b4804c1c8f27aaff5198b796"
format_stamp: "Formatted at 2026-08-09 04:40:05 on dist-test-slave-vxj2"
server_key: "945711f30a3861ca3e19ab67f05d498d"
server_key_iv: "ab65ecfaf758ecc85d7deb3ed161a1e5"
server_key_version: "encryptionkey@0"
I20260809 04:40:05.724481 6452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "d7772899b4804c1c8f27aaff5198b796"
format_stamp: "Formatted at 2026-08-09 04:40:05 on dist-test-slave-vxj2"
server_key: "945711f30a3861ca3e19ab67f05d498d"
server_key_iv: "ab65ecfaf758ecc85d7deb3ed161a1e5"
server_key_version: "encryptionkey@0"
I20260809 04:40:05.728355 6452 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:40:05.730767 6470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:05.732033 6452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:40:05.732128 6452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "d7772899b4804c1c8f27aaff5198b796"
format_stamp: "Formatted at 2026-08-09 04:40:05 on dist-test-slave-vxj2"
server_key: "945711f30a3861ca3e19ab67f05d498d"
server_key_iv: "ab65ecfaf758ecc85d7deb3ed161a1e5"
server_key_version: "encryptionkey@0"
I20260809 04:40:05.732213 6452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:05.750554 6452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:05.753665 6452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:05.753823 6452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:05.754352 6452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:05.755208 6452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:05.755254 6452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:05.755292 6452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:05.755311 6452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:05.765674 6452 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:33667
I20260809 04:40:05.765734 6604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:33667 every 8 connection(s)
I20260809 04:40:05.766669 6452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:05.775197 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 6452
I20260809 04:40:05.775333 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:40:05.775655 6011 external_mini_cluster.cc:1428] Setting key be7d3bd920124be01433814dda7763a7
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250405, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:05.780674 6116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:05.768517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:45753 (local address 127.5.222.254:42231)
0809 04:40:05.768684 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:05.768689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:05.769611 (+ 922us) server_negotiation.cc:407] Connection header received
0809 04:40:05.771077 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:05.771080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:05.771127 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:05.771203 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:05.772388 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.772909 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.774162 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.774390 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.774477 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:05.777351 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:05.777354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:05.777357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:05.777389 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:05.778896 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:05.779370 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:05.779372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:05.779373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:05.779455 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:05.779745 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:05.779747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:05.779749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:05.779930 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:05.780005 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:05.780462 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 04:40:05.780557 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:40:05.781519 6607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:05.768861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:42231 (local address 127.5.222.195:45753)
0809 04:40:05.769375 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:05.769417 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:05.770886 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:05.771366 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:05.771372 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:05.771699 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:05.772256 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:05.772276 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.773011 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.773014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:05.774053 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:05.774062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.774519 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.774522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:05.774656 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:05.775280 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:05.775306 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:05.777149 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:05.779010 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:05.779015 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:05.779025 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:05.779280 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:05.779531 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:05.779533 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:05.779534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:05.779663 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:05.780015 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:05.780020 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:05.780243 (+ 223us) client_negotiation.cc:769] Sending connection context
0809 04:40:05.780402 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 04:40:05.780638 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":113,"threads_started":1}
I20260809 04:40:05.782829 6605 heartbeater.cc:347] Connected to a master server at 127.5.222.254:42231
I20260809 04:40:05.783087 6605 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:05.783555 6605 heartbeater.cc:511] Master 127.5.222.254:42231 requested a full tablet report, sending...
I20260809 04:40:05.784493 6072 ts_manager.cc:194] Registered new tserver with Master: d7772899b4804c1c8f27aaff5198b796 (127.5.222.195:33667)
I20260809 04:40:05.785045 6072 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:45753
I20260809 04:40:05.790364 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250405, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250405, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.222.193@KRBTEST.COM
I20260809 04:40:05.820230 6612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:05.809758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33228 (local address 127.5.222.193:46097)
0809 04:40:05.810214 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:05.810223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:05.810258 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:40:05.810433 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:05.810440 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:05.810654 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:05.810943 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:05.811649 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.812428 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.813331 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.813713 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.813828 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:05.816223 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:05.816232 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:05.816243 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:05.816266 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:05.817955 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:05.818505 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:05.818511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:05.818512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:05.818603 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:05.819079 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:05.819086 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:05.819088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:05.819403 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:05.819652 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:05.819868 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 04:40:05.820051 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":170,"threads_started":1}
W20260809 04:40:05.821198 6190 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33228
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.193@KRBTEST.COM
I20260809 04:40:05.849082 6612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:05.840891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.5.222.193:46097)
0809 04:40:05.841048 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:05.841052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:05.841084 (+ 32us) server_negotiation.cc:407] Connection header received
0809 04:40:05.841293 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:05.841296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:05.841339 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:05.841407 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:05.842174 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.842675 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.843490 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:05.843705 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:05.843778 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:05.846208 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:05.846211 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:05.846214 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:05.846235 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:05.847512 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:05.847939 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:05.847941 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:05.847942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:05.848013 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:05.848305 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:05.848307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:05.848308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:05.848458 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:05.848558 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:05.848744 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 04:40:05.848834 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:40:05.850431 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 6139
I20260809 04:40:05.857344 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 6295
I20260809 04:40:05.863669 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 6452
I20260809 04:40:05.871480 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 6033
2026-08-09T04:40:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2371 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6621](info): set up 2 sockets
Aug 09 04:40:05 dist-test-slave-vxj2 krb5kdc[6621](info): commencing operation
krb5kdc: starting...
W20260809 04:40:06.972823 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:07 dist-test-slave-vxj2 krb5kdc[6621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:40:07.098039 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:35409
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:33387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:35409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:07.210287 6634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:07.210615 6634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:07.210691 6634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:07.214781 6634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:07.214880 6634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:07.214919 6634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:07.214959 6634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:07.214989 6634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:07.220299 6634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:33387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:35409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:35409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:07.221438 6634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:07.222494 6634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:07.231142 6641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:07.231191 6643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:07.231142 6639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:07.231583 6634 server_base.cc:1034] running on GCE node
I20260809 04:40:07.232146 6634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:07.233333 6634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:07.234548 6634 hybrid_clock.cc:648] HybridClock initialized: now 1786250407234527 us; error 30 us; skew 500 ppm
Aug 09 04:40:07 dist-test-slave-vxj2 krb5kdc[6621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250407, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:07.237540 6634 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:40:07.238492 6634 webserver.cc:460] Webserver started at http://127.5.222.254:38629/ using document root <none> and password file <none>
I20260809 04:40:07.239032 6634 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:07.239091 6634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:07.239267 6634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:07.241459 6634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "15a9cfd9c2ef4972a41f56d0d62c393e"
format_stamp: "Formatted at 2026-08-09 04:40:07 on dist-test-slave-vxj2"
server_key: "ac69dfc9d21e096b319fb27d87b5ff6b"
server_key_iv: "78fb267541b067671aa096c9a9398b16"
server_key_version: "encryptionkey@0"
I20260809 04:40:07.242072 6634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "15a9cfd9c2ef4972a41f56d0d62c393e"
format_stamp: "Formatted at 2026-08-09 04:40:07 on dist-test-slave-vxj2"
server_key: "ac69dfc9d21e096b319fb27d87b5ff6b"
server_key_iv: "78fb267541b067671aa096c9a9398b16"
server_key_version: "encryptionkey@0"
I20260809 04:40:07.246158 6634 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:40:07.248646 6650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:07.250047 6634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:40:07.250182 6634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "15a9cfd9c2ef4972a41f56d0d62c393e"
format_stamp: "Formatted at 2026-08-09 04:40:07 on dist-test-slave-vxj2"
server_key: "ac69dfc9d21e096b319fb27d87b5ff6b"
server_key_iv: "78fb267541b067671aa096c9a9398b16"
server_key_version: "encryptionkey@0"
I20260809 04:40:07.250293 6634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:07.268644 6634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:07.271711 6634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:07.271922 6634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:07.279256 6634 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:35409
I20260809 04:40:07.279299 6711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:35409 every 8 connection(s)
I20260809 04:40:07.280217 6634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:07.283069 6712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:07.284271 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 6634
I20260809 04:40:07.284390 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:07.284679 6011 external_mini_cluster.cc:1428] Setting key 8643f5e3f83423411bb59857ad9fd541
I20260809 04:40:07.289630 6712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e: Bootstrap starting.
I20260809 04:40:07.292491 6712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:07.293488 6712 log.cc:824] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:40:07 dist-test-slave-vxj2 krb5kdc[6621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:07.295522 6712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e: No bootstrap required, opened a new log
I20260809 04:40:07.299216 6715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:07.286089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36714 (local address 127.5.222.254:35409)
0809 04:40:07.286516 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:07.286531 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:07.286566 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:40:07.287176 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:07.287191 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:07.287506 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:07.287771 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:07.288650 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:07.289742 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:07.290966 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:07.291430 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:07.291549 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:07.294649 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:07.294655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:07.294667 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:07.294693 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:07.296432 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:07.296886 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:07.296891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:07.296895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:07.297021 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:07.297339 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:07.297343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:07.297344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:07.297727 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:07.297928 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:07.298234 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 04:40:07.298512 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":135,"threads_started":1}
I20260809 04:40:07.299305 6712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a9cfd9c2ef4972a41f56d0d62c393e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35409 } }
I20260809 04:40:07.299589 6712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:07.299676 6712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a9cfd9c2ef4972a41f56d0d62c393e, State: Initialized, Role: FOLLOWER
I20260809 04:40:07.300112 6712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a9cfd9c2ef4972a41f56d0d62c393e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35409 } }
I20260809 04:40:07.300262 6712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:07.300346 6712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:07.300472 6712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:07.301520 6712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a9cfd9c2ef4972a41f56d0d62c393e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35409 } }
I20260809 04:40:07.301862 6712 leader_election.cc:302] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15a9cfd9c2ef4972a41f56d0d62c393e; no voters:
I20260809 04:40:07.302112 6712 leader_election.cc:288] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:07.302251 6717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:07.302513 6717 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [term 1 LEADER]: Becoming Leader. State: Replica: 15a9cfd9c2ef4972a41f56d0d62c393e, State: Running, Role: LEADER
I20260809 04:40:07.302824 6717 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a9cfd9c2ef4972a41f56d0d62c393e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35409 } }
I20260809 04:40:07.303431 6712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:07.304528 6719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15a9cfd9c2ef4972a41f56d0d62c393e. Latest consensus state: current_term: 1 leader_uuid: "15a9cfd9c2ef4972a41f56d0d62c393e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a9cfd9c2ef4972a41f56d0d62c393e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35409 } } }
I20260809 04:40:07.304673 6719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:07.305073 6728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:07.305034 6718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15a9cfd9c2ef4972a41f56d0d62c393e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a9cfd9c2ef4972a41f56d0d62c393e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35409 } } }
I20260809 04:40:07.305768 6718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:07.308226 6728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:07.314271 6728 catalog_manager.cc:1347] Generated new cluster ID: 4ecf78e0b33749bd9dd4fc6a74627340
I20260809 04:40:07.314344 6728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:07.320799 6728 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:07.322093 6728 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:07.329293 6728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15a9cfd9c2ef4972a41f56d0d62c393e: Generated new TSK 0
I20260809 04:40:07.330165 6728 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:40:07.366482 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35409
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:33387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:07.485455 6742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:07.485692 6742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:07.485734 6742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:07.489941 6742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:07.489995 6742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:07.490085 6742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:07.495189 6742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:33387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35409
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:07.496204 6742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:07.497221 6742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:07.507495 6752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:07.507488 6750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:07.507488 6749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:07.507640 6742 server_base.cc:1034] running on GCE node
I20260809 04:40:07.508198 6742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:07.508841 6742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:07.510026 6742 hybrid_clock.cc:648] HybridClock initialized: now 1786250407510012 us; error 27 us; skew 500 ppm
Aug 09 04:40:07 dist-test-slave-vxj2 krb5kdc[6621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250407, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:07.512718 6742 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:40:07.513625 6742 webserver.cc:460] Webserver started at http://127.5.222.193:46519/ using document root <none> and password file <none>
I20260809 04:40:07.514189 6742 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:07.514282 6742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:07.514519 6742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:07.516594 6742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "85c3889095d741acb688ebd696b9e833"
format_stamp: "Formatted at 2026-08-09 04:40:07 on dist-test-slave-vxj2"
server_key: "611d9f75caf0929c5de02208656b2c0c"
server_key_iv: "727e2ed67b726decdeda967ebcc8d3d2"
server_key_version: "encryptionkey@0"
I20260809 04:40:07.517227 6742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "85c3889095d741acb688ebd696b9e833"
format_stamp: "Formatted at 2026-08-09 04:40:07 on dist-test-slave-vxj2"
server_key: "611d9f75caf0929c5de02208656b2c0c"
server_key_iv: "727e2ed67b726decdeda967ebcc8d3d2"
server_key_version: "encryptionkey@0"
I20260809 04:40:07.520867 6742 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:40:07.523329 6759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:07.524585 6742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:40:07.524729 6742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "85c3889095d741acb688ebd696b9e833"
format_stamp: "Formatted at 2026-08-09 04:40:07 on dist-test-slave-vxj2"
server_key: "611d9f75caf0929c5de02208656b2c0c"
server_key_iv: "727e2ed67b726decdeda967ebcc8d3d2"
server_key_version: "encryptionkey@0"
I20260809 04:40:07.524838 6742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:07.550942 6742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:07.554363 6742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:07.554569 6742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:07.555250 6742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:07.556103 6742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:07.556176 6742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:07.556247 6742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:07.556304 6742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:07.565342 6742 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:32971
I20260809 04:40:07.565375 6892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:32971 every 8 connection(s)
I20260809 04:40:07.566198 6742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:07.569146 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 6742
I20260809 04:40:07.569293 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:07.569550 6011 external_mini_cluster.cc:1428] Setting key 4b37b55fe0dab8b677ca08224f410626
Aug 09 04:40:07 dist-test-slave-vxj2 krb5kdc[6621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250407, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
I20260809 04:40:07.580399 6715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:07.567934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:39897 (local address 127.5.222.254:35409)
0809 04:40:07.568078 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:07.568082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:07.568926 (+ 844us) server_negotiation.cc:407] Connection header received
0809 04:40:07.570392 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:07.570396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:07.570450 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:07.570559 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:07.571659 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:07.572184 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:07.573454 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:07.573700 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:07.573810 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:07.576519 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:07.576522 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:07.576524 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:07.576547 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:07.578222 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:07.578760 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:07.578764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:07.578766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:07.578850 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:07.579268 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:07.579272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:07.579274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:07.579464 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:07.579575 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:07.580101 (+ 526us) server_negotiation.cc:299] Negotiation successful
0809 04:40:07.580225 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:40:07.581274 6896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:07.568311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:35409 (local address 127.5.222.193:39897)
0809 04:40:07.568734 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:07.568820 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:07.570187 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:07.570724 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:07.570730 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:07.571072 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:07.571519 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:07.571531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:07.572336 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:07.572340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:07.573324 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:07.573334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:07.573853 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:07.573857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:07.574028 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:07.574631 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:07.574646 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:07.576351 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:07.578358 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:07.578364 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:07.578377 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:07.578649 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:07.578965 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:07.578968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:07.578970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:07.579168 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:07.579575 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:07.579582 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:07.579826 (+ 244us) client_negotiation.cc:769] Sending connection context
0809 04:40:07.580057 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 04:40:07.580374 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:40:07.582638 6894 heartbeater.cc:347] Connected to a master server at 127.5.222.254:35409
I20260809 04:40:07.582880 6894 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:07.583379 6894 heartbeater.cc:511] Master 127.5.222.254:35409 requested a full tablet report, sending...
I20260809 04:40:07.584899 6670 ts_manager.cc:194] Registered new tserver with Master: 85c3889095d741acb688ebd696b9e833 (127.5.222.193:32971)
I20260809 04:40:07.586076 6670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:39897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:40:07.598592 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35409
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:33387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:07.715018 6899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:07.715301 6899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:07.715345 6899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:07.719441 6899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:07.719501 6899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:07.719560 6899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:07.725013 6899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:33387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35409
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:07.726097 6899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:07.727134 6899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:07.737197 6906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:07.737198 6904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:07.737174 6908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:07.737660 6899 server_base.cc:1034] running on GCE node
I20260809 04:40:07.738037 6899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:07.738683 6899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:07.739848 6899 hybrid_clock.cc:648] HybridClock initialized: now 1786250407739822 us; error 35 us; skew 500 ppm
Aug 09 04:40:07 dist-test-slave-vxj2 krb5kdc[6621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250407, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:07.742686 6899 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:40:07.743557 6899 webserver.cc:460] Webserver started at http://127.5.222.194:40271/ using document root <none> and password file <none>
I20260809 04:40:07.744115 6899 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:07.744174 6899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:07.744424 6899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:07.746639 6899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "cb510e95b96d4b51b41652586199ef1e"
format_stamp: "Formatted at 2026-08-09 04:40:07 on dist-test-slave-vxj2"
server_key: "a59c9b54c7b1b669850d369080f677e1"
server_key_iv: "b3d3e544abfd56f55e2e34074ecf4993"
server_key_version: "encryptionkey@0"
I20260809 04:40:07.747215 6899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "cb510e95b96d4b51b41652586199ef1e"
format_stamp: "Formatted at 2026-08-09 04:40:07 on dist-test-slave-vxj2"
server_key: "a59c9b54c7b1b669850d369080f677e1"
server_key_iv: "b3d3e544abfd56f55e2e34074ecf4993"
server_key_version: "encryptionkey@0"
I20260809 04:40:07.750980 6899 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:40:07.753396 6915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:07.754703 6899 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:07.754804 6899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "cb510e95b96d4b51b41652586199ef1e"
format_stamp: "Formatted at 2026-08-09 04:40:07 on dist-test-slave-vxj2"
server_key: "a59c9b54c7b1b669850d369080f677e1"
server_key_iv: "b3d3e544abfd56f55e2e34074ecf4993"
server_key_version: "encryptionkey@0"
I20260809 04:40:07.754942 6899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:07.777577 6899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:07.780977 6899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:07.781219 6899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:07.781852 6899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:07.782707 6899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:07.782770 6899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:07.782817 6899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:07.782876 6899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:07.792721 6899 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:38563
I20260809 04:40:07.792752 7052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:38563 every 8 connection(s)
I20260809 04:40:07.793705 6899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:07.795894 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 6899
I20260809 04:40:07.796041 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:07.796334 6011 external_mini_cluster.cc:1428] Setting key 8fb6b17eed9b9c43af271cbaaadc5dcb
Aug 09 04:40:07 dist-test-slave-vxj2 krb5kdc[6621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250407, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
I20260809 04:40:07.809357 6715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:07.795783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:51547 (local address 127.5.222.254:35409)
0809 04:40:07.795925 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:07.795930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:07.796937 (+ 1007us) server_negotiation.cc:407] Connection header received
0809 04:40:07.798330 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:07.798335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:07.798394 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:07.798488 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:07.800061 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:07.800911 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:07.802119 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:07.802342 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:07.802455 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:07.805554 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:07.805564 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:07.805567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:07.805590 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:07.807051 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:07.807665 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:07.807668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:07.807669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:07.807775 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:07.808184 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:07.808187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:07.808189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:07.808367 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:07.808473 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:07.809047 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 04:40:07.809192 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
I20260809 04:40:07.810185 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:07.796270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:35409 (local address 127.5.222.194:51547)
0809 04:40:07.796773 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:07.796827 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:07.798120 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:07.798690 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:07.798700 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:07.799164 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:07.799878 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:07.799902 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:07.801042 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:07.801045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:07.801976 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:07.801985 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:07.802491 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:07.802496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:07.802658 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:07.803293 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:07.803308 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:07.805316 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:07.807191 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:07.807196 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:07.807207 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:07.807560 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:07.807931 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:07.807934 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:07.807936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:07.808079 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:07.808470 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:07.808477 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:07.808741 (+ 264us) client_negotiation.cc:769] Sending connection context
0809 04:40:07.808923 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 04:40:07.809274 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":170,"threads_started":1}
I20260809 04:40:07.811407 7053 heartbeater.cc:347] Connected to a master server at 127.5.222.254:35409
I20260809 04:40:07.811645 7053 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:07.812161 7053 heartbeater.cc:511] Master 127.5.222.254:35409 requested a full tablet report, sending...
I20260809 04:40:07.813333 6670 ts_manager.cc:194] Registered new tserver with Master: cb510e95b96d4b51b41652586199ef1e (127.5.222.194:38563)
I20260809 04:40:07.814271 6670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:51547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:40:07.828598 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35409
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:33387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:07.944991 7058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:07.945345 7058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:07.945405 7058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:07.949477 7058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:07.949539 7058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:07.949601 7058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:07.955152 7058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:33387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35409
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:07.956158 7058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:07.957131 7058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:07.966928 7064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:07.966950 7068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:07.966938 7065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:07.967247 7058 server_base.cc:1034] running on GCE node
I20260809 04:40:07.967760 7058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:07.968396 7058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:07.969580 7058 hybrid_clock.cc:648] HybridClock initialized: now 1786250407969564 us; error 33 us; skew 500 ppm
Aug 09 04:40:07 dist-test-slave-vxj2 krb5kdc[6621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250407, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:07.972292 7058 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:40:07.973140 7058 webserver.cc:460] Webserver started at http://127.5.222.195:37789/ using document root <none> and password file <none>
I20260809 04:40:07.973758 7058 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:07.973888 7058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:07.974110 7058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:07.976132 7058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "ed9a898ef88f4630ae418787e03911a6"
format_stamp: "Formatted at 2026-08-09 04:40:07 on dist-test-slave-vxj2"
server_key: "8043a87e572be0489be647c9f0409b4f"
server_key_iv: "2a30aa5219413b688a887b2ba7b50c0f"
server_key_version: "encryptionkey@0"
I20260809 04:40:07.976688 7058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "ed9a898ef88f4630ae418787e03911a6"
format_stamp: "Formatted at 2026-08-09 04:40:07 on dist-test-slave-vxj2"
server_key: "8043a87e572be0489be647c9f0409b4f"
server_key_iv: "2a30aa5219413b688a887b2ba7b50c0f"
server_key_version: "encryptionkey@0"
I20260809 04:40:07.980298 7058 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260809 04:40:07.982674 7079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:07.983798 7058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:40:07.983932 7058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "ed9a898ef88f4630ae418787e03911a6"
format_stamp: "Formatted at 2026-08-09 04:40:07 on dist-test-slave-vxj2"
server_key: "8043a87e572be0489be647c9f0409b4f"
server_key_iv: "2a30aa5219413b688a887b2ba7b50c0f"
server_key_version: "encryptionkey@0"
I20260809 04:40:07.984038 7058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:07.996562 7058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:07.999537 7058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:07.999744 7058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:08.000345 7058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:08.001183 7058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:08.001250 7058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:08.001322 7058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:08.001355 7058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:08.010684 7058 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:46631
I20260809 04:40:08.010723 7214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:46631 every 8 connection(s)
I20260809 04:40:08.011605 7058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:08.015246 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 7058
I20260809 04:40:08.015390 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:40:08.015671 6011 external_mini_cluster.cc:1428] Setting key aa6982547d01ca62b1cc6de3da6ab165
Aug 09 04:40:08 dist-test-slave-vxj2 krb5kdc[6621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250407, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:08.025519 6715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.013485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:38739 (local address 127.5.222.254:35409)
0809 04:40:08.013634 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:08.013637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:08.014471 (+ 834us) server_negotiation.cc:407] Connection header received
0809 04:40:08.015972 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:08.015975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:08.016032 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:08.016135 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:08.017518 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.018157 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.019281 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.019488 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.019563 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.022079 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:08.022081 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:08.022083 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:08.022108 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:08.023600 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:08.024080 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:08.024082 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:08.024084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:08.024157 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:08.024456 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:08.024458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:08.024459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:08.024623 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:08.024696 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:08.025284 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 04:40:08.025396 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:40:08.026331 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.013864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:35409 (local address 127.5.222.195:38739)
0809 04:40:08.014297 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:08.014365 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:08.015759 (+ 1394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:08.016311 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:08.016317 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:08.016754 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:08.017370 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:08.017389 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.018268 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.018271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:08.019173 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:08.019182 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.019611 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.019614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:08.019750 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.020359 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:08.020375 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:08.021923 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:08.023715 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:08.023720 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:08.023732 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:08.023990 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:08.024234 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:08.024236 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:08.024237 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:08.024364 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:08.024731 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:08.024737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:08.025010 (+ 273us) client_negotiation.cc:769] Sending connection context
0809 04:40:08.025232 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 04:40:08.025497 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":106,"threads_started":1}
I20260809 04:40:08.027624 7216 heartbeater.cc:347] Connected to a master server at 127.5.222.254:35409
I20260809 04:40:08.027900 7216 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:08.028558 7216 heartbeater.cc:511] Master 127.5.222.254:35409 requested a full tablet report, sending...
I20260809 04:40:08.029719 6670 ts_manager.cc:194] Registered new tserver with Master: ed9a898ef88f4630ae418787e03911a6 (127.5.222.195:46631)
I20260809 04:40:08.030314 6670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:38739
I20260809 04:40:08.041673 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:40:08.052960 6715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.045396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36730 (local address 127.5.222.254:35409)
0809 04:40:08.045530 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:08.045533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:08.045772 (+ 239us) server_negotiation.cc:407] Connection header received
0809 04:40:08.045873 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:08.045875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:08.045923 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:08.045995 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:08.046727 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.047275 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.048100 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.048339 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.048429 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.049892 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:08.049895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:08.049897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:08.049922 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:08.051239 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:08.051660 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:08.051662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:08.051664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:08.051791 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:08.052057 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:08.052060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:08.052061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:08.052300 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:08.052413 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:08.052623 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 04:40:08.052719 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:40:08.057806 6670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:08.061018 6670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:08.075984 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.070010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:32971 (local address 127.0.0.1:42334)
0809 04:40:08.070431 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:08.070487 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:08.070619 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:08.071382 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:08.071388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:08.071409 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:08.071546 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:08.071553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.072987 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.072991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:08.074354 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:08.074365 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.075436 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.075440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:08.075546 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.075583 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 04:40:08.075691 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 04:40:08.075817 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":122,"threads_started":1}
I20260809 04:40:08.076336 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.070234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.5.222.193:32971)
0809 04:40:08.070632 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:08.070653 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:08.070682 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:40:08.070766 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:08.070772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:08.070989 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:08.071153 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:08.071690 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.072829 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.074494 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.075302 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.075429 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.075581 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:08.076075 (+ 494us) server_negotiation.cc:299] Negotiation successful
0809 04:40:08.076187 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":118,"threads_started":1}
I20260809 04:40:08.076385 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.070851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33578 (local address 127.5.222.195:46631)
0809 04:40:08.071395 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:08.071408 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:08.071798 (+ 390us) server_negotiation.cc:407] Connection header received
0809 04:40:08.071886 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:08.071893 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:08.072071 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:08.072220 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:08.072449 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.073421 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.074766 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.075479 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.075579 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.075707 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:08.075890 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 04:40:08.075994 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":153,"threads_started":1}
I20260809 04:40:08.075994 7234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.070413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:46631 (local address 127.0.0.1:33578)
0809 04:40:08.071585 (+ 1172us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:08.071599 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:08.071693 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:08.072205 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:08.072208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:08.072226 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:08.072334 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:08.072343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.073572 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.073575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:08.074641 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:08.074650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.075662 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.075669 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:08.075752 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.075771 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 04:40:08.075842 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 04:40:08.075894 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1096,"thread_start_us":112,"threads_started":1}
I20260809 04:40:08.077541 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.070998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45924 (local address 127.5.222.194:38563)
0809 04:40:08.071398 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:08.071409 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:08.071831 (+ 422us) server_negotiation.cc:407] Connection header received
0809 04:40:08.071940 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:08.071945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:08.072121 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:08.072258 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:08.072629 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.073712 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.075112 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.075947 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.076048 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.076172 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:08.077275 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0809 04:40:08.077378 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":111,"threads_started":1}
I20260809 04:40:08.077549 7233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.070787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:38563 (local address 127.0.0.1:45924)
0809 04:40:08.071736 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:08.071750 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:08.071839 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:08.072389 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:08.072393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:08.072414 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:08.072507 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:08.072511 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.073869 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.073873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:08.074974 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:08.074984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.076244 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.076248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:08.076335 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.076357 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:40:08.077349 (+ 992us) client_negotiation.cc:240] Negotiation successful
0809 04:40:08.077425 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"thread_start_us":95,"threads_started":1}
I20260809 04:40:08.079361 7134 tablet_service.cc:1467] Processing CreateTablet for tablet d529d3179bbb47129d77e9fb6cc96281 (DEFAULT_TABLE table=test-table [id=f8d686aa85954a21bfe1234aabb66971]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:08.079406 6815 tablet_service.cc:1467] Processing CreateTablet for tablet d529d3179bbb47129d77e9fb6cc96281 (DEFAULT_TABLE table=test-table [id=f8d686aa85954a21bfe1234aabb66971]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:08.080374 7134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d529d3179bbb47129d77e9fb6cc96281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:08.080461 6815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d529d3179bbb47129d77e9fb6cc96281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:08.080132 6974 tablet_service.cc:1467] Processing CreateTablet for tablet d529d3179bbb47129d77e9fb6cc96281 (DEFAULT_TABLE table=test-table [id=f8d686aa85954a21bfe1234aabb66971]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:08.081079 6974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d529d3179bbb47129d77e9fb6cc96281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:08.087325 7237 tablet_bootstrap.cc:492] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6: Bootstrap starting.
I20260809 04:40:08.087967 7240 tablet_bootstrap.cc:492] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e: Bootstrap starting.
I20260809 04:40:08.088466 7239 tablet_bootstrap.cc:492] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833: Bootstrap starting.
I20260809 04:40:08.090242 7239 tablet_bootstrap.cc:654] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:08.090775 7240 tablet_bootstrap.cc:654] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:08.091240 7239 log.cc:824] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:08.091384 7240 log.cc:824] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:08.093309 7239 tablet_bootstrap.cc:492] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833: No bootstrap required, opened a new log
I20260809 04:40:08.093532 7239 ts_tablet_manager.cc:1397] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 04:40:08.093675 7237 tablet_bootstrap.cc:654] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:08.094758 7240 tablet_bootstrap.cc:492] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e: No bootstrap required, opened a new log
I20260809 04:40:08.094735 7237 log.cc:824] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:08.094961 7240 ts_tablet_manager.cc:1397] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 04:40:08.096884 7237 tablet_bootstrap.cc:492] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6: No bootstrap required, opened a new log
I20260809 04:40:08.097064 7237 ts_tablet_manager.cc:1397] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 04:40:08.098140 7239 raft_consensus.cc:348] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85c3889095d741acb688ebd696b9e833" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32971 } } peers { permanent_uuid: "cb510e95b96d4b51b41652586199ef1e" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 38563 } } peers { permanent_uuid: "ed9a898ef88f4630ae418787e03911a6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46631 } }
I20260809 04:40:08.098495 7239 raft_consensus.cc:374] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:08.098562 7239 raft_consensus.cc:729] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85c3889095d741acb688ebd696b9e833, State: Initialized, Role: FOLLOWER
I20260809 04:40:08.098418 7240 raft_consensus.cc:348] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85c3889095d741acb688ebd696b9e833" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32971 } } peers { permanent_uuid: "cb510e95b96d4b51b41652586199ef1e" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 38563 } } peers { permanent_uuid: "ed9a898ef88f4630ae418787e03911a6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46631 } }
I20260809 04:40:08.098696 7240 raft_consensus.cc:374] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:08.098753 7240 raft_consensus.cc:729] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb510e95b96d4b51b41652586199ef1e, State: Initialized, Role: FOLLOWER
I20260809 04:40:08.099154 7240 consensus_queue.cc:260] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85c3889095d741acb688ebd696b9e833" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32971 } } peers { permanent_uuid: "cb510e95b96d4b51b41652586199ef1e" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 38563 } } peers { permanent_uuid: "ed9a898ef88f4630ae418787e03911a6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46631 } }
I20260809 04:40:08.099156 7239 consensus_queue.cc:260] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85c3889095d741acb688ebd696b9e833" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32971 } } peers { permanent_uuid: "cb510e95b96d4b51b41652586199ef1e" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 38563 } } peers { permanent_uuid: "ed9a898ef88f4630ae418787e03911a6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46631 } }
I20260809 04:40:08.099786 7053 heartbeater.cc:503] Master 127.5.222.254:35409 was elected leader, sending a full tablet report...
I20260809 04:40:08.100140 7240 ts_tablet_manager.cc:1428] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 04:40:08.100220 7239 ts_tablet_manager.cc:1428] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 04:40:08.100541 6894 heartbeater.cc:503] Master 127.5.222.254:35409 was elected leader, sending a full tablet report...
I20260809 04:40:08.103282 7237 raft_consensus.cc:348] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85c3889095d741acb688ebd696b9e833" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32971 } } peers { permanent_uuid: "cb510e95b96d4b51b41652586199ef1e" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 38563 } } peers { permanent_uuid: "ed9a898ef88f4630ae418787e03911a6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46631 } }
I20260809 04:40:08.103550 7237 raft_consensus.cc:374] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:08.103616 7237 raft_consensus.cc:729] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9a898ef88f4630ae418787e03911a6, State: Initialized, Role: FOLLOWER
I20260809 04:40:08.104097 7237 consensus_queue.cc:260] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85c3889095d741acb688ebd696b9e833" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32971 } } peers { permanent_uuid: "cb510e95b96d4b51b41652586199ef1e" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 38563 } } peers { permanent_uuid: "ed9a898ef88f4630ae418787e03911a6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46631 } }
I20260809 04:40:08.104842 7216 heartbeater.cc:503] Master 127.5.222.254:35409 was elected leader, sending a full tablet report...
I20260809 04:40:08.105158 7237 ts_tablet_manager.cc:1428] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 04:40:08.133561 7245 raft_consensus.cc:482] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:08.133805 7245 raft_consensus.cc:504] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85c3889095d741acb688ebd696b9e833" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32971 } } peers { permanent_uuid: "cb510e95b96d4b51b41652586199ef1e" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 38563 } } peers { permanent_uuid: "ed9a898ef88f4630ae418787e03911a6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46631 } }
I20260809 04:40:08.134948 7245 leader_election.cc:288] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb510e95b96d4b51b41652586199ef1e (127.5.222.194:38563), ed9a898ef88f4630ae418787e03911a6 (127.5.222.195:46631)
I20260809 04:40:08.140638 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.135424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:55855 (local address 127.5.222.194:38563)
0809 04:40:08.135563 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:08.135569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:08.135834 (+ 265us) server_negotiation.cc:407] Connection header received
0809 04:40:08.135976 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:08.135980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:08.136038 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:08.136135 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:08.136540 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.137446 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.139246 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.139959 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.140078 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.140132 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:08.140425 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 04:40:08.140501 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:40:08.140638 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.135353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:56841 (local address 127.5.222.195:46631)
0809 04:40:08.135516 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:08.135521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:08.135907 (+ 386us) server_negotiation.cc:407] Connection header received
0809 04:40:08.135977 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:08.135980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:08.136038 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:08.136133 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:08.136540 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.137455 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.139248 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.139939 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.140078 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.140132 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:08.140425 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 04:40:08.140501 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 04:40:08.140702 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.135258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:46631 (local address 127.5.222.193:56841)
0809 04:40:08.135787 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:08.135800 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:08.135891 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:08.136157 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:08.136160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:08.136199 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:08.136408 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:08.136416 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.137572 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.137575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:08.139099 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:08.139107 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.140081 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.140085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:08.140267 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.140305 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 04:40:08.140387 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 04:40:08.140508 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":280,"spinlock_wait_cycles":28672,"thread_start_us":112,"threads_started":1}
I20260809 04:40:08.140955 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.135247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:38563 (local address 127.5.222.193:55855)
0809 04:40:08.135724 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:08.135740 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:08.135864 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:08.136157 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:08.136160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:08.136200 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:08.136412 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:08.136425 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.137572 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.137576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:08.139096 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:08.139107 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.140081 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.140085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:08.140262 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.140315 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 04:40:08.140387 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 04:40:08.140508 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":219,"spinlock_wait_cycles":6912,"thread_start_us":154,"threads_started":1}
I20260809 04:40:08.141448 7156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d529d3179bbb47129d77e9fb6cc96281" candidate_uuid: "85c3889095d741acb688ebd696b9e833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9a898ef88f4630ae418787e03911a6" is_pre_election: true
I20260809 04:40:08.141650 6999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d529d3179bbb47129d77e9fb6cc96281" candidate_uuid: "85c3889095d741acb688ebd696b9e833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb510e95b96d4b51b41652586199ef1e" is_pre_election: true
I20260809 04:40:08.141794 7156 raft_consensus.cc:2454] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85c3889095d741acb688ebd696b9e833 in term 0.
I20260809 04:40:08.141994 6999 raft_consensus.cc:2454] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85c3889095d741acb688ebd696b9e833 in term 0.
I20260809 04:40:08.142411 6761 leader_election.cc:302] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85c3889095d741acb688ebd696b9e833, ed9a898ef88f4630ae418787e03911a6; no voters:
I20260809 04:40:08.142691 7245 raft_consensus.cc:2781] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:08.142792 7245 raft_consensus.cc:482] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:08.142843 7245 raft_consensus.cc:3037] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:08.144073 7245 raft_consensus.cc:504] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85c3889095d741acb688ebd696b9e833" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32971 } } peers { permanent_uuid: "cb510e95b96d4b51b41652586199ef1e" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 38563 } } peers { permanent_uuid: "ed9a898ef88f4630ae418787e03911a6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46631 } }
I20260809 04:40:08.144419 7245 leader_election.cc:288] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [CANDIDATE]: Term 1 election: Requested vote from peers cb510e95b96d4b51b41652586199ef1e (127.5.222.194:38563), ed9a898ef88f4630ae418787e03911a6 (127.5.222.195:46631)
I20260809 04:40:08.144765 7156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d529d3179bbb47129d77e9fb6cc96281" candidate_uuid: "85c3889095d741acb688ebd696b9e833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9a898ef88f4630ae418787e03911a6"
I20260809 04:40:08.144791 6999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d529d3179bbb47129d77e9fb6cc96281" candidate_uuid: "85c3889095d741acb688ebd696b9e833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb510e95b96d4b51b41652586199ef1e"
I20260809 04:40:08.144919 6999 raft_consensus.cc:3037] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:08.144906 7156 raft_consensus.cc:3037] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:08.146148 6999 raft_consensus.cc:2454] T d529d3179bbb47129d77e9fb6cc96281 P cb510e95b96d4b51b41652586199ef1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85c3889095d741acb688ebd696b9e833 in term 1.
I20260809 04:40:08.146338 7156 raft_consensus.cc:2454] T d529d3179bbb47129d77e9fb6cc96281 P ed9a898ef88f4630ae418787e03911a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85c3889095d741acb688ebd696b9e833 in term 1.
I20260809 04:40:08.146605 6762 leader_election.cc:302] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85c3889095d741acb688ebd696b9e833, cb510e95b96d4b51b41652586199ef1e; no voters:
I20260809 04:40:08.146833 7245 raft_consensus.cc:2781] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:08.147089 7245 raft_consensus.cc:686] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [term 1 LEADER]: Becoming Leader. State: Replica: 85c3889095d741acb688ebd696b9e833, State: Running, Role: LEADER
I20260809 04:40:08.147481 7245 consensus_queue.cc:237] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85c3889095d741acb688ebd696b9e833" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32971 } } peers { permanent_uuid: "cb510e95b96d4b51b41652586199ef1e" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 38563 } } peers { permanent_uuid: "ed9a898ef88f4630ae418787e03911a6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46631 } }
I20260809 04:40:08.150724 6670 catalog_manager.cc:5495] T d529d3179bbb47129d77e9fb6cc96281 P 85c3889095d741acb688ebd696b9e833 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85c3889095d741acb688ebd696b9e833 (127.5.222.193). New cstate: current_term: 1 leader_uuid: "85c3889095d741acb688ebd696b9e833" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85c3889095d741acb688ebd696b9e833" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb510e95b96d4b51b41652586199ef1e" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 38563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed9a898ef88f4630ae418787e03911a6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46631 } health_report { overall_health: UNKNOWN } } }
Aug 09 04:40:08 dist-test-slave-vxj2 krb5kdc[6621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.193@KRBTEST.COM
I20260809 04:40:08.177774 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.169095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.5.222.193:32971)
0809 04:40:08.169295 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:08.169299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:08.169385 (+ 86us) server_negotiation.cc:407] Connection header received
0809 04:40:08.169505 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:08.169507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:08.169547 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:08.169614 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:08.170274 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.170809 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.171647 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.171914 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.172006 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.174549 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:08.174561 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:08.174575 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:08.174598 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:08.176123 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:08.176519 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:08.176524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:08.176526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:08.176617 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:08.176846 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:08.176849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:08.176850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:08.177089 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:08.177331 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:08.177459 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 04:40:08.177555 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
Aug 09 04:40:08 dist-test-slave-vxj2 krb5kdc[6621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250408, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:40:08 dist-test-slave-vxj2 krb5kdc[6621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250408, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.222.193@KRBTEST.COM
I20260809 04:40:08.205279 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.196882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42342 (local address 127.5.222.193:32971)
0809 04:40:08.197070 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:08.197074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:08.197145 (+ 71us) server_negotiation.cc:407] Connection header received
0809 04:40:08.197334 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:08.197336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:08.197379 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:08.197449 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:08.198190 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.198690 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.199499 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.199720 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.199799 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.202272 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:08.202275 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:08.202277 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:08.202298 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:08.203501 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:08.203880 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:08.203883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:08.203884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:08.203973 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:08.204229 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:08.204231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:08.204233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:08.204392 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:08.204528 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:08.204837 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 04:40:08.204972 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
Aug 09 04:40:08 dist-test-slave-vxj2 krb5kdc[6621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:40:08 dist-test-slave-vxj2 krb5kdc[6621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.193@KRBTEST.COM
I20260809 04:40:08.233740 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:08.224996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42350 (local address 127.5.222.193:32971)
0809 04:40:08.225185 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:08.225190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:08.225495 (+ 305us) server_negotiation.cc:407] Connection header received
0809 04:40:08.225667 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:08.225671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:08.225721 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:08.225810 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:08.226517 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.227038 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.227872 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:08.228107 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:08.228214 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:08.230812 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:08.230815 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:08.230817 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:08.230840 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:08.232232 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:08.232661 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:08.232664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:08.232665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:08.232739 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:08.232975 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:08.232977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:08.232978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:08.233135 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:08.233320 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:08.233521 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 04:40:08.233620 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 04:40:08.235219 6790 tablet_service.cc:2908] Rejecting scan request for tablet d529d3179bbb47129d77e9fb6cc96281: Uninitialized: clean time has not yet been initialized
I20260809 04:40:08.237596 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 6742
I20260809 04:40:08.246937 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 6899
I20260809 04:40:08.255261 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 7058
I20260809 04:40:08.261561 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 6634
2026-08-09T04:40:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2390 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:08 dist-test-slave-vxj2 krb5kdc[7267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:08 dist-test-slave-vxj2 krb5kdc[7267](info): set up 2 sockets
Aug 09 04:40:08 dist-test-slave-vxj2 krb5kdc[7267](info): commencing operation
krb5kdc: starting...
W20260809 04:40:09.348042 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:09 dist-test-slave-vxj2 krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:40:09.475356 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:43471
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:43471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:09.587024 7281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:09.587360 7281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:09.587443 7281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:09.591570 7281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:09.591663 7281 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 04:40:09.591694 7281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:09.591739 7281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:09.591790 7281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:09.591826 7281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:09.597337 7281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:43471
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:43471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:09.598479 7281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:09.599542 7281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:40:09.608211 7281 server_base.cc:1034] running on GCE node
W20260809 04:40:09.608130 7291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:09.608152 7289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:09.608173 7294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:09.609030 7281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:09.610180 7281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:09.611356 7281 hybrid_clock.cc:648] HybridClock initialized: now 1786250409611326 us; error 42 us; skew 500 ppm
Aug 09 04:40:09 dist-test-slave-vxj2 krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250409, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:09.614423 7281 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:40:09.615366 7281 webserver.cc:460] Webserver started at http://127.5.222.254:45791/ using document root <none> and password file <none>
I20260809 04:40:09.615901 7281 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:09.615962 7281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:09.616127 7281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:09.618245 7281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "ebe97641f6c4498ebf064fc61e8f78f2"
format_stamp: "Formatted at 2026-08-09 04:40:09 on dist-test-slave-vxj2"
server_key: "d3fec3bd19557a522c23a33c18bc10a6"
server_key_iv: "836a0474a81f17ec88948178cfc43bd9"
server_key_version: "encryptionkey@0"
I20260809 04:40:09.618784 7281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "ebe97641f6c4498ebf064fc61e8f78f2"
format_stamp: "Formatted at 2026-08-09 04:40:09 on dist-test-slave-vxj2"
server_key: "d3fec3bd19557a522c23a33c18bc10a6"
server_key_iv: "836a0474a81f17ec88948178cfc43bd9"
server_key_version: "encryptionkey@0"
I20260809 04:40:09.622514 7281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:40:09.624882 7302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:09.626250 7281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:09.626456 7281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "ebe97641f6c4498ebf064fc61e8f78f2"
format_stamp: "Formatted at 2026-08-09 04:40:09 on dist-test-slave-vxj2"
server_key: "d3fec3bd19557a522c23a33c18bc10a6"
server_key_iv: "836a0474a81f17ec88948178cfc43bd9"
server_key_version: "encryptionkey@0"
I20260809 04:40:09.626587 7281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:09.638702 7281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:09.642028 7281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:09.642235 7281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:09.650161 7281 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:43471
I20260809 04:40:09.650189 7375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:43471 every 8 connection(s)
I20260809 04:40:09.651078 7281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:09.651742 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 7281
I20260809 04:40:09.651862 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:09.652151 6011 external_mini_cluster.cc:1428] Setting key f9d4e997337f50780609891632963a8c
I20260809 04:40:09.654290 7377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:09.659582 7377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2: Bootstrap starting.
Aug 09 04:40:09 dist-test-slave-vxj2 krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:09.661842 7377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:09.663257 7377 log.cc:824] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:09.665580 7377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2: No bootstrap required, opened a new log
I20260809 04:40:09.667019 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:09.653649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46154 (local address 127.5.222.254:43471)
0809 04:40:09.654102 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:09.654117 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:09.654152 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:40:09.654870 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:09.654887 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:09.655206 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:09.655541 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:09.656333 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:09.657407 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:09.658267 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:09.658728 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:09.658900 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:09.661431 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:09.661436 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:09.661448 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:09.661473 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:09.663701 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:09.664207 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:09.664216 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:09.664225 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:09.664398 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:09.664801 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:09.664805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:09.664807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:09.665375 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:09.665599 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:09.665943 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 04:40:09.666271 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":136,"threads_started":1}
I20260809 04:40:09.668924 7377 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe97641f6c4498ebf064fc61e8f78f2" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43471 } }
I20260809 04:40:09.669212 7377 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:09.669276 7377 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebe97641f6c4498ebf064fc61e8f78f2, State: Initialized, Role: FOLLOWER
I20260809 04:40:09.669723 7377 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe97641f6c4498ebf064fc61e8f78f2" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43471 } }
I20260809 04:40:09.669845 7377 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:09.669884 7377 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:09.669956 7377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:09.670807 7377 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe97641f6c4498ebf064fc61e8f78f2" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43471 } }
I20260809 04:40:09.671089 7377 leader_election.cc:302] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebe97641f6c4498ebf064fc61e8f78f2; no voters:
I20260809 04:40:09.671368 7377 leader_election.cc:288] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:09.671448 7382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:09.671666 7382 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [term 1 LEADER]: Becoming Leader. State: Replica: ebe97641f6c4498ebf064fc61e8f78f2, State: Running, Role: LEADER
I20260809 04:40:09.672016 7382 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe97641f6c4498ebf064fc61e8f78f2" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43471 } }
I20260809 04:40:09.672667 7377 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:09.673189 7382 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebe97641f6c4498ebf064fc61e8f78f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe97641f6c4498ebf064fc61e8f78f2" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43471 } } }
I20260809 04:40:09.673385 7382 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:09.673621 7383 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebe97641f6c4498ebf064fc61e8f78f2. Latest consensus state: current_term: 1 leader_uuid: "ebe97641f6c4498ebf064fc61e8f78f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe97641f6c4498ebf064fc61e8f78f2" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43471 } } }
I20260809 04:40:09.673779 7383 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:09.673887 7389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:09.677060 7389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:09.682837 7389 catalog_manager.cc:1347] Generated new cluster ID: 8d09745ae55f4e04bec02e782224be7d
I20260809 04:40:09.682998 7389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:09.699381 7389 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:09.700657 7389 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:09.722697 7389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2: Generated new TSK 0
I20260809 04:40:09.723620 7389 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:40:09.755643 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43471
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:09.866477 7407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:09.866791 7407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:09.866873 7407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:09.870869 7407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:09.870970 7407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:09.871039 7407 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:40:09.871124 7407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:09.876758 7407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.222.254:43471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:09.877899 7407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:09.878958 7407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:09.888827 7416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:09.888799 7413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:09.888895 7414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:09.889470 7407 server_base.cc:1034] running on GCE node
I20260809 04:40:09.889966 7407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:09.890632 7407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:09.891786 7407 hybrid_clock.cc:648] HybridClock initialized: now 1786250409891773 us; error 24 us; skew 500 ppm
Aug 09 04:40:09 dist-test-slave-vxj2 krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250409, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:09.894564 7407 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:40:09.895375 7407 webserver.cc:460] Webserver started at http://127.5.222.193:42039/ using document root <none> and password file <none>
I20260809 04:40:09.895882 7407 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:09.895939 7407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:09.896092 7407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:09.898109 7407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "953d7569ba97416d960f3afcdf599679"
format_stamp: "Formatted at 2026-08-09 04:40:09 on dist-test-slave-vxj2"
server_key: "cad16213510493fc3b14525f1fca9c58"
server_key_iv: "a2e821c594ba3d7aa884928a61b5f699"
server_key_version: "encryptionkey@0"
I20260809 04:40:09.898643 7407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "953d7569ba97416d960f3afcdf599679"
format_stamp: "Formatted at 2026-08-09 04:40:09 on dist-test-slave-vxj2"
server_key: "cad16213510493fc3b14525f1fca9c58"
server_key_iv: "a2e821c594ba3d7aa884928a61b5f699"
server_key_version: "encryptionkey@0"
I20260809 04:40:09.902324 7407 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:40:09.904788 7426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:09.906034 7407 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:09.906155 7407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "953d7569ba97416d960f3afcdf599679"
format_stamp: "Formatted at 2026-08-09 04:40:09 on dist-test-slave-vxj2"
server_key: "cad16213510493fc3b14525f1fca9c58"
server_key_iv: "a2e821c594ba3d7aa884928a61b5f699"
server_key_version: "encryptionkey@0"
I20260809 04:40:09.906257 7407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:09.920187 7407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:09.923444 7407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:09.923647 7407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:09.925146 7407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:09.925267 7407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:09.925359 7407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:09.925401 7407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:40:09 dist-test-slave-vxj2 krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250409, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:09.936316 7407 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:34435
I20260809 04:40:09.936414 7559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:34435 every 8 connection(s)
I20260809 04:40:09.937445 7407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:09.940764 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:09.926098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:54849 (local address 127.5.222.254:43471)
0809 04:40:09.926267 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:09.926272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:09.927139 (+ 867us) server_negotiation.cc:407] Connection header received
0809 04:40:09.928185 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:09.928191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:09.928271 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:09.928353 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:09.930265 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:09.931286 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:09.933066 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:09.933517 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:09.933663 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:09.936918 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:09.936922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:09.936924 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:09.936951 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:09.938699 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:09.939214 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:09.939218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:09.939220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:09.939311 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:09.939663 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:09.939666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:09.939668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:09.939855 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:09.939979 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:09.940454 (+ 475us) server_negotiation.cc:299] Negotiation successful
0809 04:40:09.940589 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:40:09.941567 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:09.926519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43471 (local address 127.5.222.193:54849)
0809 04:40:09.926966 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:09.927016 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:09.927936 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:09.928831 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:09.928839 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:09.929334 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:09.930073 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:09.930091 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:09.931426 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:09.931432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:09.932919 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:09.932929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:09.933716 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:09.933719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:09.933888 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:09.934612 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:09.934639 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:09.936731 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:09.938813 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:09.938819 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:09.938832 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:09.939098 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:09.939409 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:09.939412 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:09.939421 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:09.939560 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:09.939980 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:09.939988 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:09.940222 (+ 234us) client_negotiation.cc:769] Sending connection context
0809 04:40:09.940407 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 04:40:09.940722 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":129,"threads_started":1}
I20260809 04:40:09.942544 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 7407
I20260809 04:40:09.942687 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:09.942961 7560 heartbeater.cc:347] Connected to a master server at 127.5.222.254:43471
I20260809 04:40:09.943023 6011 external_mini_cluster.cc:1428] Setting key e0fb48397b2eb9d6113e787535e0b672
I20260809 04:40:09.943135 7560 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:09.943673 7560 heartbeater.cc:511] Master 127.5.222.254:43471 requested a full tablet report, sending...
I20260809 04:40:09.945338 7322 ts_manager.cc:194] Registered new tserver with Master: 953d7569ba97416d960f3afcdf599679 (127.5.222.193:34435)
I20260809 04:40:09.946748 7322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:54849
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:40:09.956911 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:09.945287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43471 (local address 127.5.222.193:48753)
0809 04:40:09.949366 (+ 4079us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:09.949380 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:09.949469 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:09.949770 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:09.949773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:09.949955 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:09.950055 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:09.950061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:09.951220 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:09.951222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:09.951831 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:09.951840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:09.952509 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:09.952512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:09.952585 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:09.953243 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:09.953256 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:09.953647 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:09.955390 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:09.955397 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:09.955403 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:09.955619 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:09.955903 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:09.955908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:09.955915 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:09.956006 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:09.956497 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:09.956500 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:09.956576 (+ 76us) client_negotiation.cc:769] Sending connection context
0809 04:40:09.956665 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 04:40:09.956767 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3995,"thread_start_us":108,"threads_started":1}
I20260809 04:40:09.957209 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:09.946445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:48753 (local address 127.5.222.254:43471)
0809 04:40:09.946595 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:09.946598 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:09.949462 (+ 2864us) server_negotiation.cc:407] Connection header received
0809 04:40:09.949543 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:09.949549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:09.949606 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:09.949700 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:09.950179 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:09.951072 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:09.951954 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:09.952374 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:09.952515 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:09.953777 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:09.953783 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:09.953790 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:09.953812 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:09.955252 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:09.955707 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:09.955709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:09.955711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:09.955783 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:09.956119 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:09.956126 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:09.956129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:09.956376 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:09.956495 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:09.956759 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 04:40:09.956912 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:40:09.972883 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43471
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:10.084342 7572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:10.084607 7572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:10.084653 7572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:10.088716 7572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:10.088780 7572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:10.088819 7572 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:40:10.088905 7572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:10.094117 7572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.222.254:43471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:10.095166 7572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:10.096274 7572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:10.105898 7577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:10.105890 7578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:10.105890 7581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:10.106169 7572 server_base.cc:1034] running on GCE node
I20260809 04:40:10.106638 7572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:10.107306 7572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:10.108480 7572 hybrid_clock.cc:648] HybridClock initialized: now 1786250410108451 us; error 49 us; skew 500 ppm
Aug 09 04:40:10 dist-test-slave-vxj2 krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250410, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:10.115981 7572 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:40:10.116922 7572 webserver.cc:460] Webserver started at http://127.5.222.194:45389/ using document root <none> and password file <none>
I20260809 04:40:10.117581 7572 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:10.117676 7572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:10.117920 7572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:10.119998 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "153f9577f2f74c18836399fa08d91de5"
format_stamp: "Formatted at 2026-08-09 04:40:10 on dist-test-slave-vxj2"
server_key: "f2d4c37f860b5d05ffbfc149aed3aec2"
server_key_iv: "412df7ec3b74e40ba68dba806c53d52a"
server_key_version: "encryptionkey@0"
I20260809 04:40:10.120587 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "153f9577f2f74c18836399fa08d91de5"
format_stamp: "Formatted at 2026-08-09 04:40:10 on dist-test-slave-vxj2"
server_key: "f2d4c37f860b5d05ffbfc149aed3aec2"
server_key_iv: "412df7ec3b74e40ba68dba806c53d52a"
server_key_version: "encryptionkey@0"
I20260809 04:40:10.124367 7572 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:40:10.126798 7589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:10.128010 7572 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:40:10.128147 7572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "153f9577f2f74c18836399fa08d91de5"
format_stamp: "Formatted at 2026-08-09 04:40:10 on dist-test-slave-vxj2"
server_key: "f2d4c37f860b5d05ffbfc149aed3aec2"
server_key_iv: "412df7ec3b74e40ba68dba806c53d52a"
server_key_version: "encryptionkey@0"
I20260809 04:40:10.128257 7572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:10.141338 7572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:10.144377 7572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:10.144590 7572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:10.146101 7572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:10.146178 7572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:10.146261 7572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:10.146306 7572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:40:10 dist-test-slave-vxj2 krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250410, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:10.157671 7572 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:34951
I20260809 04:40:10.157748 7724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:34951 every 8 connection(s)
I20260809 04:40:10.158584 7572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:10.159535 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 7572
I20260809 04:40:10.159660 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:10.159931 6011 external_mini_cluster.cc:1428] Setting key d8fee955ac21772fd595eb6384f984e8
I20260809 04:40:10.161010 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.146914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:49501 (local address 127.5.222.254:43471)
0809 04:40:10.147041 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.147045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.148242 (+ 1197us) server_negotiation.cc:407] Connection header received
0809 04:40:10.149445 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.149449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.149516 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.149628 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:10.150999 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.151535 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.152714 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.152995 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.153116 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.156161 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:10.156166 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:10.156169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:10.156199 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:10.157989 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:10.158502 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:10.158505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:10.158507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:10.158598 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:10.158942 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:10.158945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:10.158947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:10.159188 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:10.159304 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.160775 (+ 1471us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.160893 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 04:40:10.162065 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.147485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43471 (local address 127.5.222.194:49501)
0809 04:40:10.148073 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.148128 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.149190 (+ 1062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.149794 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.149803 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.150270 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:10.150832 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.150874 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.151662 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.151665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.152583 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.152592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.153194 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.153200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.153397 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.154105 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:10.154131 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:10.155989 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:10.158106 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:10.158112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:10.158125 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:10.158382 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:10.158689 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:10.158694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:10.158696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:10.158837 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:10.160314 (+ 1477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:10.160319 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:10.160539 (+ 220us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.160746 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.161071 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":152,"threads_started":1}
I20260809 04:40:10.163472 7725 heartbeater.cc:347] Connected to a master server at 127.5.222.254:43471
I20260809 04:40:10.163623 7725 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:10.164227 7725 heartbeater.cc:511] Master 127.5.222.254:43471 requested a full tablet report, sending...
I20260809 04:40:10.165344 7322 ts_manager.cc:194] Registered new tserver with Master: 153f9577f2f74c18836399fa08d91de5 (127.5.222.194:34951)
I20260809 04:40:10.166133 7322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:49501
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
I20260809 04:40:10.174302 7733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.165878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43471 (local address 127.5.222.194:56615)
0809 04:40:10.166181 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.166196 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.166311 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.166754 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.166760 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.167014 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:10.167186 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.167197 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.168187 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.168194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.169269 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.169283 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.169734 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.169740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.169870 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.170369 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:10.170389 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:10.170825 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:10.172530 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:10.172534 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:10.172536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:10.172811 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:10.173195 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:10.173199 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:10.173201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:10.173288 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:10.173833 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:10.173836 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:10.173949 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.174057 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.174167 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":95,"threads_started":1}
I20260809 04:40:10.174376 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.166131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:56615 (local address 127.5.222.254:43471)
0809 04:40:10.166240 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.166243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.166258 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:40:10.166473 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.166478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.166535 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.166594 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:10.167328 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.168050 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.169405 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.169614 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.169692 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.170988 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:10.170991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:10.170993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:10.171014 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:10.172419 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:10.172904 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:10.172906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:10.172907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:10.172993 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:10.173399 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:10.173401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:10.173402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:10.173647 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:10.173783 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.174149 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.174270 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:40:10.191872 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43471
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:10.315485 7735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:10.315760 7735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:10.315858 7735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:10.319940 7735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:10.320020 7735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:10.320084 7735 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:40:10.320165 7735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:10.325299 7735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.222.254:43471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:10.326267 7735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:10.327252 7735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:10.336634 7740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:10.336673 7741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:10.336827 7735 server_base.cc:1034] running on GCE node
W20260809 04:40:10.336694 7744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:10.337347 7735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:10.338079 7735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:10.339296 7735 hybrid_clock.cc:648] HybridClock initialized: now 1786250410339281 us; error 34 us; skew 500 ppm
Aug 09 04:40:10 dist-test-slave-vxj2 krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250410, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:10.342201 7735 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:40:10.343050 7735 webserver.cc:460] Webserver started at http://127.5.222.195:33463/ using document root <none> and password file <none>
I20260809 04:40:10.343650 7735 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:10.343737 7735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:10.343969 7735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:10.346118 7735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "8da4562a895a474b81211cefb02b4a43"
format_stamp: "Formatted at 2026-08-09 04:40:10 on dist-test-slave-vxj2"
server_key: "563073a7ae444ae6ce7fb819975a9984"
server_key_iv: "30bab022c7c904b00d8fc951dddb1dbd"
server_key_version: "encryptionkey@0"
I20260809 04:40:10.346694 7735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "8da4562a895a474b81211cefb02b4a43"
format_stamp: "Formatted at 2026-08-09 04:40:10 on dist-test-slave-vxj2"
server_key: "563073a7ae444ae6ce7fb819975a9984"
server_key_iv: "30bab022c7c904b00d8fc951dddb1dbd"
server_key_version: "encryptionkey@0"
I20260809 04:40:10.350414 7735 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:40:10.352694 7754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:10.353868 7735 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:10.354009 7735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "8da4562a895a474b81211cefb02b4a43"
format_stamp: "Formatted at 2026-08-09 04:40:10 on dist-test-slave-vxj2"
server_key: "563073a7ae444ae6ce7fb819975a9984"
server_key_iv: "30bab022c7c904b00d8fc951dddb1dbd"
server_key_version: "encryptionkey@0"
I20260809 04:40:10.354120 7735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:10.367014 7735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:10.369683 7735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:10.369889 7735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:10.371243 7735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:10.371315 7735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:10.371385 7735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:10.371421 7735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:40:10 dist-test-slave-vxj2 krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250410, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:10.382423 7735 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:41149
I20260809 04:40:10.382529 7892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:41149 every 8 connection(s)
I20260809 04:40:10.383311 7735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:10.385718 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.372085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:55311 (local address 127.5.222.254:43471)
0809 04:40:10.372219 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.372223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.373349 (+ 1126us) server_negotiation.cc:407] Connection header received
0809 04:40:10.374542 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.374546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.374608 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.374697 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:10.376283 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.377148 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.378422 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.378834 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.378955 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.381686 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:10.381693 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:10.381699 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:10.381736 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:10.383487 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:10.384111 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:10.384115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:10.384117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:10.384217 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:10.384540 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:10.384542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:10.384543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:10.384743 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:10.384822 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.385497 (+ 675us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.385602 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:40:10.386646 7765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.372658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43471 (local address 127.5.222.195:55311)
0809 04:40:10.373209 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.373250 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.374323 (+ 1073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.374901 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.374914 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.375402 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:10.376100 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.376122 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.377306 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.377310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.378281 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.378290 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.379046 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.379051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.379256 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.379885 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:10.379904 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:10.381514 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:10.383618 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:10.383623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:10.383636 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:10.383990 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:10.384312 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:10.384315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:10.384316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:10.384451 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:10.384885 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:10.384890 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:10.385230 (+ 340us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.385443 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.385712 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":137,"threads_started":1}
I20260809 04:40:10.387915 7894 heartbeater.cc:347] Connected to a master server at 127.5.222.254:43471
I20260809 04:40:10.388048 7894 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:10.388195 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 7735
I20260809 04:40:10.388358 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:40:10.388598 7894 heartbeater.cc:511] Master 127.5.222.254:43471 requested a full tablet report, sending...
I20260809 04:40:10.388659 6011 external_mini_cluster.cc:1428] Setting key 7c1a598d846e60cce4559233bd70b3ae
I20260809 04:40:10.390216 7321 ts_manager.cc:194] Registered new tserver with Master: 8da4562a895a474b81211cefb02b4a43 (127.5.222.195:41149)
I20260809 04:40:10.391026 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:40:10.391090 7321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:55311
I20260809 04:40:10.398285 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.390180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:45991 (local address 127.5.222.254:43471)
0809 04:40:10.390292 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.390296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.390622 (+ 326us) server_negotiation.cc:407] Connection header received
0809 04:40:10.390725 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.390728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.390775 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.390856 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:10.391500 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.392043 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.392953 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.393296 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.393418 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.394728 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:10.394731 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:10.394733 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:10.394754 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:10.396324 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:10.396937 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:10.396939 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:10.396941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:10.397014 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:10.397388 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:10.397390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:10.397392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:10.397551 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:10.397626 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.398054 (+ 428us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.398166 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:40:10.398757 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.390219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43471 (local address 127.5.222.195:45991)
0809 04:40:10.390521 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.390536 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.390644 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.390970 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.390973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.391213 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:10.391344 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.391356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.392181 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.392185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.392822 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.392830 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.393423 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.393427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.393526 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.394007 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:10.394018 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:10.394372 (+ 354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:10.396492 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:10.396496 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:10.396508 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:10.396809 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:10.397119 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:10.397126 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:10.397131 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:10.397278 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:10.397652 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:10.397659 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:10.397796 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.398465 (+ 669us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.398601 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":110,"threads_started":1}
Aug 09 04:40:10 dist-test-slave-vxj2 krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250410, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:40:10 dist-test-slave-vxj2 krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250410, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:10.418396 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.410114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46162 (local address 127.5.222.254:43471)
0809 04:40:10.410230 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.410234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.410249 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:40:10.410288 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.410290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.410326 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.410398 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:10.411164 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.411678 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.412535 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.412783 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.412864 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.415334 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:10.415337 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:10.415340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:10.415366 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:10.416731 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:10.417143 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:10.417145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:10.417146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:10.417294 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:10.417544 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:10.417546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:10.417547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:10.417717 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:10.417899 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.418151 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.418270 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:40:10.421869 7321 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:10.425383 7321 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:10.440487 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.435106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51132 (local address 127.5.222.193:34435)
0809 04:40:10.436046 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.436061 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.436081 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:40:10.436150 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.436155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.436323 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.436469 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:10.436730 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.437809 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.439071 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.439719 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.439850 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.440036 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.440235 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.440341 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":74,"threads_started":1}
I20260809 04:40:10.440477 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.433934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:34951 (local address 127.0.0.1:50802)
0809 04:40:10.434695 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.434752 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.434951 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.435644 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.435648 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.435668 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:10.435768 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.435772 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.437135 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.437139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.438780 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.438792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.439938 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.439945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.440038 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.440067 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.440153 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.440312 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":27,"thread_start_us":78,"threads_started":1}
I20260809 04:40:10.440750 7918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.434540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35286 (local address 127.5.222.195:41149)
0809 04:40:10.435029 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.435042 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.435313 (+ 271us) server_negotiation.cc:407] Connection header received
0809 04:40:10.435785 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.435796 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.435939 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.436097 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:10.436359 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.437511 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.439155 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.439911 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.440007 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.440164 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.440494 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.440594 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":124,"threads_started":1}
I20260809 04:40:10.441047 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.434655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:34435 (local address 127.0.0.1:51132)
0809 04:40:10.435454 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.435465 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.435545 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.436479 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.436482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.436503 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:10.436607 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.436614 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.437943 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.437946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.438930 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.438941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.439860 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.439864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.440008 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.440062 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.440209 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.440889 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":32,"thread_start_us":102,"threads_started":1}
I20260809 04:40:10.441093 7919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.434112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:41149 (local address 127.0.0.1:35286)
0809 04:40:10.435208 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.435223 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.435636 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.436087 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.436090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.436109 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:10.436241 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.436247 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.437656 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.437660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.438960 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.438968 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.440223 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.440226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.440322 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.440341 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.440881 (+ 540us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.440960 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":33,"thread_start_us":104,"threads_started":1}
I20260809 04:40:10.441377 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.434048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50802 (local address 127.5.222.194:34951)
0809 04:40:10.434457 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.434469 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.434839 (+ 370us) server_negotiation.cc:407] Connection header received
0809 04:40:10.435142 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.435149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.435370 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.435482 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:10.435884 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.437012 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.438921 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.439775 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.440308 (+ 533us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.440468 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.440514 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.441229 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":104,"threads_started":1}
I20260809 04:40:10.444069 7813 tablet_service.cc:1467] Processing CreateTablet for tablet f0446fba97054c2086bbc1681c11cd67 (DEFAULT_TABLE table=test-table [id=77afb0581f474d88adc6c2cd4103c2c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:10.444169 7484 tablet_service.cc:1467] Processing CreateTablet for tablet f0446fba97054c2086bbc1681c11cd67 (DEFAULT_TABLE table=test-table [id=77afb0581f474d88adc6c2cd4103c2c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:10.444182 7651 tablet_service.cc:1467] Processing CreateTablet for tablet f0446fba97054c2086bbc1681c11cd67 (DEFAULT_TABLE table=test-table [id=77afb0581f474d88adc6c2cd4103c2c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:10.445218 7813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0446fba97054c2086bbc1681c11cd67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:10.445267 7484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0446fba97054c2086bbc1681c11cd67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:10.445267 7651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0446fba97054c2086bbc1681c11cd67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:10.451695 7922 tablet_bootstrap.cc:492] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43: Bootstrap starting.
I20260809 04:40:10.453250 7923 tablet_bootstrap.cc:492] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679: Bootstrap starting.
I20260809 04:40:10.453641 7924 tablet_bootstrap.cc:492] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5: Bootstrap starting.
I20260809 04:40:10.454171 7922 tablet_bootstrap.cc:654] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:10.455102 7922 log.cc:824] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:10.455780 7924 tablet_bootstrap.cc:654] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:10.456377 7923 tablet_bootstrap.cc:654] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:10.456589 7924 log.cc:824] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:10.457244 7922 tablet_bootstrap.cc:492] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43: No bootstrap required, opened a new log
I20260809 04:40:10.457307 7923 log.cc:824] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:10.457438 7922 ts_tablet_manager.cc:1397] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:40:10.458778 7924 tablet_bootstrap.cc:492] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5: No bootstrap required, opened a new log
I20260809 04:40:10.459007 7924 ts_tablet_manager.cc:1397] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:40:10.459285 7923 tablet_bootstrap.cc:492] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679: No bootstrap required, opened a new log
I20260809 04:40:10.459493 7923 ts_tablet_manager.cc:1397] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:40:10.461436 7922 raft_consensus.cc:348] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } }
I20260809 04:40:10.461807 7922 raft_consensus.cc:374] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:10.461906 7922 raft_consensus.cc:729] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da4562a895a474b81211cefb02b4a43, State: Initialized, Role: FOLLOWER
I20260809 04:40:10.462457 7922 consensus_queue.cc:260] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } }
I20260809 04:40:10.462831 7924 raft_consensus.cc:348] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } }
I20260809 04:40:10.463100 7924 raft_consensus.cc:374] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:10.463195 7924 raft_consensus.cc:729] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153f9577f2f74c18836399fa08d91de5, State: Initialized, Role: FOLLOWER
I20260809 04:40:10.463104 7923 raft_consensus.cc:348] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } }
I20260809 04:40:10.463418 7923 raft_consensus.cc:374] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:10.463483 7923 raft_consensus.cc:729] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953d7569ba97416d960f3afcdf599679, State: Initialized, Role: FOLLOWER
I20260809 04:40:10.463534 7922 ts_tablet_manager.cc:1428] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:40:10.463878 7894 heartbeater.cc:503] Master 127.5.222.254:43471 was elected leader, sending a full tablet report...
I20260809 04:40:10.463958 7923 consensus_queue.cc:260] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } }
I20260809 04:40:10.463665 7924 consensus_queue.cc:260] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } }
I20260809 04:40:10.464604 7560 heartbeater.cc:503] Master 127.5.222.254:43471 was elected leader, sending a full tablet report...
I20260809 04:40:10.464874 7923 ts_tablet_manager.cc:1428] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 04:40:10.465368 7924 ts_tablet_manager.cc:1428] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:40:10.465538 7725 heartbeater.cc:503] Master 127.5.222.254:43471 was elected leader, sending a full tablet report...
I20260809 04:40:10.541726 7928 raft_consensus.cc:482] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:10.542007 7928 raft_consensus.cc:504] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } }
I20260809 04:40:10.543282 7928 leader_election.cc:288] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 153f9577f2f74c18836399fa08d91de5 (127.5.222.194:34951), 953d7569ba97416d960f3afcdf599679 (127.5.222.193:34435)
I20260809 04:40:10.547169 7765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.543527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:34951 (local address 127.5.222.195:39911)
0809 04:40:10.543710 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.543725 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.543843 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.544180 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.544185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.544206 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:10.544350 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.544357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.545131 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.545133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.546160 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.546170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.546841 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.546844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.546934 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.546954 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.547013 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.547066 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 04:40:10.547242 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.543613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:39911 (local address 127.5.222.194:34951)
0809 04:40:10.543723 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.543728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.543839 (+ 111us) server_negotiation.cc:407] Connection header received
0809 04:40:10.543961 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.543963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.544037 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.544146 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:10.544472 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.545016 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.546286 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.546702 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.546840 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.546892 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.547050 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.547115 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:40:10.547897 7673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0446fba97054c2086bbc1681c11cd67" candidate_uuid: "8da4562a895a474b81211cefb02b4a43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153f9577f2f74c18836399fa08d91de5" is_pre_election: true
I20260809 04:40:10.548045 7931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.543562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:34435 (local address 127.5.222.195:37199)
0809 04:40:10.543902 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.543915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.544005 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.544218 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.544220 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.544244 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:10.544860 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.544868 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.545725 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.545728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.546684 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.546692 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.547632 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.547638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.547733 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.547766 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.547845 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.547910 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":142,"threads_started":1}
I20260809 04:40:10.548038 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.543658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:37199 (local address 127.5.222.193:34435)
0809 04:40:10.543788 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.543792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.544003 (+ 211us) server_negotiation.cc:407] Connection header received
0809 04:40:10.544081 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.544087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.544143 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.544212 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:10.544984 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.545606 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.546809 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.547394 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.547524 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.547574 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.547867 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.547926 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:40:10.548231 7673 raft_consensus.cc:2454] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da4562a895a474b81211cefb02b4a43 in term 0.
I20260809 04:40:10.548591 7506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0446fba97054c2086bbc1681c11cd67" candidate_uuid: "8da4562a895a474b81211cefb02b4a43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953d7569ba97416d960f3afcdf599679" is_pre_election: true
I20260809 04:40:10.548836 7758 leader_election.cc:302] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 153f9577f2f74c18836399fa08d91de5, 8da4562a895a474b81211cefb02b4a43; no voters:
I20260809 04:40:10.548905 7506 raft_consensus.cc:2454] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da4562a895a474b81211cefb02b4a43 in term 0.
I20260809 04:40:10.549098 7928 raft_consensus.cc:2781] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:10.549224 7928 raft_consensus.cc:482] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:10.549293 7928 raft_consensus.cc:3037] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:10.550333 7928 raft_consensus.cc:504] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } }
I20260809 04:40:10.550673 7928 leader_election.cc:288] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [CANDIDATE]: Term 1 election: Requested vote from peers 153f9577f2f74c18836399fa08d91de5 (127.5.222.194:34951), 953d7569ba97416d960f3afcdf599679 (127.5.222.193:34435)
I20260809 04:40:10.550961 7673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0446fba97054c2086bbc1681c11cd67" candidate_uuid: "8da4562a895a474b81211cefb02b4a43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153f9577f2f74c18836399fa08d91de5"
I20260809 04:40:10.551106 7673 raft_consensus.cc:3037] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:10.551146 7506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0446fba97054c2086bbc1681c11cd67" candidate_uuid: "8da4562a895a474b81211cefb02b4a43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953d7569ba97416d960f3afcdf599679"
I20260809 04:40:10.551251 7506 raft_consensus.cc:3037] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:10.552594 7506 raft_consensus.cc:2454] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da4562a895a474b81211cefb02b4a43 in term 1.
I20260809 04:40:10.552925 7756 leader_election.cc:302] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da4562a895a474b81211cefb02b4a43, 953d7569ba97416d960f3afcdf599679; no voters:
I20260809 04:40:10.553073 7673 raft_consensus.cc:2454] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da4562a895a474b81211cefb02b4a43 in term 1.
I20260809 04:40:10.553136 7928 raft_consensus.cc:2781] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:10.553371 7928 raft_consensus.cc:686] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 1 LEADER]: Becoming Leader. State: Replica: 8da4562a895a474b81211cefb02b4a43, State: Running, Role: LEADER
I20260809 04:40:10.553773 7928 consensus_queue.cc:237] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } }
I20260809 04:40:10.556972 7321 catalog_manager.cc:5495] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da4562a895a474b81211cefb02b4a43 (127.5.222.195). New cstate: current_term: 1 leader_uuid: "8da4562a895a474b81211cefb02b4a43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260809 04:40:10.596983 7918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.593100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35298 (local address 127.5.222.195:41149)
0809 04:40:10.593285 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.593290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.593305 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:40:10.593535 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.593537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.593591 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.593701 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:10.593992 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.594531 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.595488 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.595711 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.595822 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.596165 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:10.596563 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:10.596690 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.596775 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.596833 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:40:10.602841 7673 raft_consensus.cc:1261] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5 [term 1 FOLLOWER]: Refusing update from remote peer 8da4562a895a474b81211cefb02b4a43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:10.602846 7506 raft_consensus.cc:1261] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679 [term 1 FOLLOWER]: Refusing update from remote peer 8da4562a895a474b81211cefb02b4a43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:10.603546 7932 consensus_queue.cc:1035] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:10.603757 7928 consensus_queue.cc:1035] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:10.613703 7938 mvcc.cc:205] Tried to move back new op lower bound from 7316481681821843456 to 7316481681630978048. Current Snapshot: MvccSnapshot[applied={T|T < 7316481681821843456 or (T in {7316481681821843456})}]
I20260809 04:40:10.622639 7322 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:10.622850 7322 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:10.625540 7322 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2: Sending DeleteTablet for 3 replicas of tablet f0446fba97054c2086bbc1681c11cd67
I20260809 04:40:10.626399 7813 tablet_service.cc:1514] Processing DeleteTablet for tablet f0446fba97054c2086bbc1681c11cd67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:10 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:35286
I20260809 04:40:10.626544 7484 tablet_service.cc:1514] Processing DeleteTablet for tablet f0446fba97054c2086bbc1681c11cd67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:10 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:51132
I20260809 04:40:10.626581 7651 tablet_service.cc:1514] Processing DeleteTablet for tablet f0446fba97054c2086bbc1681c11cd67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:10 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:50802
I20260809 04:40:10.626967 7948 tablet_replica.cc:330] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679: stopping tablet replica
I20260809 04:40:10.627259 7947 tablet_replica.cc:330] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43: stopping tablet replica
I20260809 04:40:10.627310 7948 raft_consensus.cc:2229] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:10.627635 7948 raft_consensus.cc:2258] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:10.627651 7947 raft_consensus.cc:2229] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:10.627947 7322 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 04:40:10.628281 7947 raft_consensus.cc:2258] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 04:40:10.628392 7322 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:10.628750 7949 tablet_replica.cc:330] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5: stopping tablet replica
I20260809 04:40:10.629109 7949 raft_consensus.cc:2229] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:10.629396 7948 ts_tablet_manager.cc:1905] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:10.629573 7949 raft_consensus.cc:2258] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:10.631464 7947 ts_tablet_manager.cc:1905] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:10.632440 7948 ts_tablet_manager.cc:1918] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:10.632567 7948 log.cc:1196] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/wal/wals/f0446fba97054c2086bbc1681c11cd67
I20260809 04:40:10.632931 7948 ts_tablet_manager.cc:1939] T f0446fba97054c2086bbc1681c11cd67 P 953d7569ba97416d960f3afcdf599679: Deleting consensus metadata
I20260809 04:40:10.634130 7651 tablet_service.cc:1467] Processing CreateTablet for tablet 8bc75f53e79a45be872a62de301e58e8 (DEFAULT_TABLE table=test-table [id=5de5051125654301a8fcff0be787a120]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:10.634436 7651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bc75f53e79a45be872a62de301e58e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:10.634749 7947 ts_tablet_manager.cc:1918] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:10.634846 7947 log.cc:1196] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/f0446fba97054c2086bbc1681c11cd67
I20260809 04:40:10.635161 7947 ts_tablet_manager.cc:1939] T f0446fba97054c2086bbc1681c11cd67 P 8da4562a895a474b81211cefb02b4a43: Deleting consensus metadata
I20260809 04:40:10.636574 7303 catalog_manager.cc:4841] TS 953d7569ba97416d960f3afcdf599679 (127.5.222.193:34435): tablet f0446fba97054c2086bbc1681c11cd67 (table test-table [id=77afb0581f474d88adc6c2cd4103c2c0]) successfully deleted
I20260809 04:40:10.636620 7924 tablet_bootstrap.cc:492] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5: Bootstrap starting.
I20260809 04:40:10.637027 7306 catalog_manager.cc:4841] TS 8da4562a895a474b81211cefb02b4a43 (127.5.222.195:41149): tablet f0446fba97054c2086bbc1681c11cd67 (table test-table [id=77afb0581f474d88adc6c2cd4103c2c0]) successfully deleted
I20260809 04:40:10.637269 7484 tablet_service.cc:1467] Processing CreateTablet for tablet 8bc75f53e79a45be872a62de301e58e8 (DEFAULT_TABLE table=test-table [id=5de5051125654301a8fcff0be787a120]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:10.637514 7484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bc75f53e79a45be872a62de301e58e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:10.637650 7924 tablet_bootstrap.cc:654] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:10.638249 7949 ts_tablet_manager.cc:1905] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:10.638520 7813 tablet_service.cc:1467] Processing CreateTablet for tablet 8bc75f53e79a45be872a62de301e58e8 (DEFAULT_TABLE table=test-table [id=5de5051125654301a8fcff0be787a120]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:10.638753 7813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bc75f53e79a45be872a62de301e58e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:10.640630 7923 tablet_bootstrap.cc:492] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679: Bootstrap starting.
I20260809 04:40:10.641129 7922 tablet_bootstrap.cc:492] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43: Bootstrap starting.
I20260809 04:40:10.641628 7923 tablet_bootstrap.cc:654] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:10.642120 7924 tablet_bootstrap.cc:492] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5: No bootstrap required, opened a new log
I20260809 04:40:10.642203 7924 ts_tablet_manager.cc:1397] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260809 04:40:10.642343 7922 tablet_bootstrap.cc:654] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:10.642652 7949 ts_tablet_manager.cc:1918] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:10.642760 7949 log.cc:1196] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/f0446fba97054c2086bbc1681c11cd67
I20260809 04:40:10.642803 7924 raft_consensus.cc:348] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.642958 7924 raft_consensus.cc:374] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:10.643021 7924 raft_consensus.cc:729] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153f9577f2f74c18836399fa08d91de5, State: Initialized, Role: FOLLOWER
I20260809 04:40:10.643177 7949 ts_tablet_manager.cc:1939] T f0446fba97054c2086bbc1681c11cd67 P 153f9577f2f74c18836399fa08d91de5: Deleting consensus metadata
I20260809 04:40:10.643880 7923 tablet_bootstrap.cc:492] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679: No bootstrap required, opened a new log
I20260809 04:40:10.643971 7923 ts_tablet_manager.cc:1397] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:10.643169 7924 consensus_queue.cc:260] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.644429 7923 raft_consensus.cc:348] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.644539 7923 raft_consensus.cc:374] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:10.644577 7923 raft_consensus.cc:729] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953d7569ba97416d960f3afcdf599679, State: Initialized, Role: FOLLOWER
I20260809 04:40:10.644693 7923 consensus_queue.cc:260] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.644806 7305 catalog_manager.cc:4841] TS 153f9577f2f74c18836399fa08d91de5 (127.5.222.194:34951): tablet f0446fba97054c2086bbc1681c11cd67 (table test-table [id=77afb0581f474d88adc6c2cd4103c2c0]) successfully deleted
I20260809 04:40:10.644949 7923 ts_tablet_manager.cc:1428] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:10.645298 7924 ts_tablet_manager.cc:1428] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260809 04:40:10.646175 7922 tablet_bootstrap.cc:492] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43: No bootstrap required, opened a new log
I20260809 04:40:10.646281 7922 ts_tablet_manager.cc:1397] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260809 04:40:10.646744 7922 raft_consensus.cc:348] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.646852 7922 raft_consensus.cc:374] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:10.646889 7922 raft_consensus.cc:729] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da4562a895a474b81211cefb02b4a43, State: Initialized, Role: FOLLOWER
I20260809 04:40:10.647094 7922 consensus_queue.cc:260] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.647835 7922 ts_tablet_manager.cc:1428] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20260809 04:40:10.659802 7726 tablet.cc:2369] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:40:10.689435 7562 tablet.cc:2369] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:40:10.777509 7929 raft_consensus.cc:482] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:10.777678 7929 raft_consensus.cc:504] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.778920 7929 leader_election.cc:288] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 153f9577f2f74c18836399fa08d91de5 (127.5.222.194:34951), 8da4562a895a474b81211cefb02b4a43 (127.5.222.195:41149)
I20260809 04:40:10.782799 7930 raft_consensus.cc:482] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:10.782935 7930 raft_consensus.cc:504] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.783463 7918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.779337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:43103 (local address 127.5.222.195:41149)
0809 04:40:10.779472 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.779477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.779717 (+ 240us) server_negotiation.cc:407] Connection header received
0809 04:40:10.779944 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.779947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.780012 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.780113 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:10.780483 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.781058 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.782249 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.782703 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.782829 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.782896 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.783255 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.783336 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 04:40:10.783872 7953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.779206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:41149 (local address 127.5.222.193:43103)
0809 04:40:10.779605 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.779621 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.779750 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.780171 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.780174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.780186 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:10.780320 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.780326 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.781210 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.781216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.782115 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.782124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.782986 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.782989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.783094 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.783118 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.783653 (+ 535us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.783730 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":102,"threads_started":1}
I20260809 04:40:10.784138 7930 leader_election.cc:288] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 953d7569ba97416d960f3afcdf599679 (127.5.222.193:34435), 8da4562a895a474b81211cefb02b4a43 (127.5.222.195:41149)
I20260809 04:40:10.784495 7837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc75f53e79a45be872a62de301e58e8" candidate_uuid: "953d7569ba97416d960f3afcdf599679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da4562a895a474b81211cefb02b4a43" is_pre_election: true
I20260809 04:40:10.784723 7837 raft_consensus.cc:2454] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953d7569ba97416d960f3afcdf599679 in term 0.
I20260809 04:40:10.784794 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.779390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:56093 (local address 127.5.222.194:34951)
0809 04:40:10.779523 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.779527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.780028 (+ 501us) server_negotiation.cc:407] Connection header received
0809 04:40:10.780159 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.780163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.780212 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.780319 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:10.780621 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.781459 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.783137 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.783753 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.783836 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.783879 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.784605 (+ 726us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.784681 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:40:10.785475 7430 leader_election.cc:302] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da4562a895a474b81211cefb02b4a43, 953d7569ba97416d960f3afcdf599679; no voters:
I20260809 04:40:10.785784 7929 raft_consensus.cc:2781] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:10.785869 7929 raft_consensus.cc:482] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:10.785801 7954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.779206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:34951 (local address 127.5.222.193:56093)
0809 04:40:10.779919 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.779934 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.780049 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.780335 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.780339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.780357 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:10.780485 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.780492 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.781595 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.781598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.782851 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.782860 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.784342 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.784346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.784461 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.784482 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.785482 (+ 1000us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.785554 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":332,"thread_start_us":97,"threads_started":1}
I20260809 04:40:10.785939 7929 raft_consensus.cc:3037] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:10.787789 7929 raft_consensus.cc:504] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.788146 7929 leader_election.cc:288] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [CANDIDATE]: Term 1 election: Requested vote from peers 153f9577f2f74c18836399fa08d91de5 (127.5.222.194:34951), 8da4562a895a474b81211cefb02b4a43 (127.5.222.195:41149)
I20260809 04:40:10.788749 7673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc75f53e79a45be872a62de301e58e8" candidate_uuid: "953d7569ba97416d960f3afcdf599679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153f9577f2f74c18836399fa08d91de5" is_pre_election: true
I20260809 04:40:10.788887 7673 raft_consensus.cc:2454] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953d7569ba97416d960f3afcdf599679 in term 0.
I20260809 04:40:10.789258 7837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc75f53e79a45be872a62de301e58e8" candidate_uuid: "953d7569ba97416d960f3afcdf599679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da4562a895a474b81211cefb02b4a43"
I20260809 04:40:10.789374 7837 raft_consensus.cc:3037] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:10.789767 7673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc75f53e79a45be872a62de301e58e8" candidate_uuid: "953d7569ba97416d960f3afcdf599679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153f9577f2f74c18836399fa08d91de5"
I20260809 04:40:10.789880 7673 raft_consensus.cc:3037] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:10.790051 7918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.785285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:58211 (local address 127.5.222.195:41149)
0809 04:40:10.785406 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.785409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.785641 (+ 232us) server_negotiation.cc:407] Connection header received
0809 04:40:10.785824 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.785827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.785876 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.785927 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:10.786299 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.787052 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.788720 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.789355 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.789441 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.789496 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.789848 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.789930 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:40:10.790278 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.784748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:49697 (local address 127.5.222.193:34435)
0809 04:40:10.785049 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.785054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.785083 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:40:10.785123 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.785126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.785189 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.785255 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:10.786293 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.787054 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.788841 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.789448 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.789537 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.789583 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.790106 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.790183 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229}
I20260809 04:40:10.790428 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.785122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:41149 (local address 127.5.222.194:58211)
0809 04:40:10.785567 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.785580 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.785705 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.786049 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.786052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.786098 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:10.786172 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.786177 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.787198 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.787201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.788548 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.788558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.789589 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.789595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.789711 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.789735 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.790166 (+ 431us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.790251 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":7,"spinlock_wait_cycles":3328,"thread_start_us":79,"threads_started":1}
I20260809 04:40:10.790642 7837 raft_consensus.cc:2454] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953d7569ba97416d960f3afcdf599679 in term 1.
I20260809 04:40:10.790894 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.784441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:34435 (local address 127.5.222.194:49697)
0809 04:40:10.784906 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.784923 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.785013 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.786010 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.786015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.786036 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:10.786167 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.786174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.787198 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.787201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.788711 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.788719 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.789762 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.789768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.789927 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.789984 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.790652 (+ 668us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.790733 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":268,"spinlock_wait_cycles":20352,"thread_start_us":72,"threads_started":1}
I20260809 04:40:10.790946 7430 leader_election.cc:302] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da4562a895a474b81211cefb02b4a43, 953d7569ba97416d960f3afcdf599679; no voters:
I20260809 04:40:10.791128 7673 raft_consensus.cc:2454] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953d7569ba97416d960f3afcdf599679 in term 1.
I20260809 04:40:10.791169 7929 raft_consensus.cc:2781] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:10.791338 7506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc75f53e79a45be872a62de301e58e8" candidate_uuid: "153f9577f2f74c18836399fa08d91de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953d7569ba97416d960f3afcdf599679" is_pre_election: true
I20260809 04:40:10.791393 7929 raft_consensus.cc:686] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 1 LEADER]: Becoming Leader. State: Replica: 953d7569ba97416d960f3afcdf599679, State: Running, Role: LEADER
I20260809 04:40:10.791442 7837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc75f53e79a45be872a62de301e58e8" candidate_uuid: "153f9577f2f74c18836399fa08d91de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da4562a895a474b81211cefb02b4a43" is_pre_election: true
I20260809 04:40:10.791594 7837 raft_consensus.cc:2379] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 153f9577f2f74c18836399fa08d91de5 in current term 1: Already voted for candidate 953d7569ba97416d960f3afcdf599679 in this term.
I20260809 04:40:10.791674 7929 consensus_queue.cc:237] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.793341 7590 leader_election.cc:302] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 153f9577f2f74c18836399fa08d91de5; no voters: 8da4562a895a474b81211cefb02b4a43, 953d7569ba97416d960f3afcdf599679
I20260809 04:40:10.793629 7930 raft_consensus.cc:2726] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 04:40:10.794292 7320 catalog_manager.cc:5495] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 953d7569ba97416d960f3afcdf599679 (127.5.222.193). New cstate: current_term: 1 leader_uuid: "953d7569ba97416d960f3afcdf599679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } health_report { overall_health: UNKNOWN } } }
Aug 09 04:40:10 dist-test-slave-vxj2 krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250409, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:10.835700 7968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.826858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43471 (local address 127.0.0.1:46168)
0809 04:40:10.827296 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:10.827311 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:10.827420 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:10.827663 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:10.827665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:10.828100 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:10.828200 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.828206 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.828911 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.828913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.829533 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:10.829541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.829989 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.829992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:10.830072 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.830692 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:10.830706 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:10.832273 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:10.834169 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:10.834178 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:10.834183 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:10.834437 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:10.834686 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:10.834688 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:10.834690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:10.834805 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:10.835176 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:10.835183 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:10.835292 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 04:40:10.835392 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 04:40:10.835526 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":206,"threads_started":1}
I20260809 04:40:10.835711 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:10.826991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46168 (local address 127.5.222.254:43471)
0809 04:40:10.827126 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:10.827130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:10.827385 (+ 255us) server_negotiation.cc:407] Connection header received
0809 04:40:10.827504 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:10.827506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:10.827545 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:10.827627 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:10.828308 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.828811 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.829672 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:10.829880 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:10.829987 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:10.832411 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:10.832414 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:10.832416 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:10.832437 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:10.834045 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:10.834527 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:10.834530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:10.834531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:10.834602 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:10.834896 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:10.834899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:10.834900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:10.835062 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:10.835135 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:10.835447 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 04:40:10.835582 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":52}
I20260809 04:40:10.840363 7320 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:46168:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 04:40:10.841128 7320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:10.846015 7651 tablet_service.cc:1467] Processing CreateTablet for tablet 58147cdd2e6641e8b18fea7610498bd6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=43847e01a8e74f5d936982a95f75e392]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:40:10.846010 7484 tablet_service.cc:1467] Processing CreateTablet for tablet 58147cdd2e6641e8b18fea7610498bd6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=43847e01a8e74f5d936982a95f75e392]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:40:10.846091 7813 tablet_service.cc:1467] Processing CreateTablet for tablet 58147cdd2e6641e8b18fea7610498bd6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=43847e01a8e74f5d936982a95f75e392]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:40:10.846405 7813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58147cdd2e6641e8b18fea7610498bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:10.846405 7484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58147cdd2e6641e8b18fea7610498bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:10.846405 7651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58147cdd2e6641e8b18fea7610498bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:10.848855 7924 tablet_bootstrap.cc:492] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5: Bootstrap starting.
I20260809 04:40:10.849130 7922 tablet_bootstrap.cc:492] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43: Bootstrap starting.
I20260809 04:40:10.849202 7923 tablet_bootstrap.cc:492] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: Bootstrap starting.
I20260809 04:40:10.850080 7922 tablet_bootstrap.cc:654] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:10.850188 7923 tablet_bootstrap.cc:654] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:10.850276 7924 tablet_bootstrap.cc:654] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:10.851643 7923 tablet_bootstrap.cc:492] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: No bootstrap required, opened a new log
I20260809 04:40:10.851737 7924 tablet_bootstrap.cc:492] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5: No bootstrap required, opened a new log
I20260809 04:40:10.851738 7923 ts_tablet_manager.cc:1397] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:10.851826 7924 ts_tablet_manager.cc:1397] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:10.852025 7922 tablet_bootstrap.cc:492] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43: No bootstrap required, opened a new log
I20260809 04:40:10.852113 7922 ts_tablet_manager.cc:1397] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 04:40:10.852223 7923 raft_consensus.cc:348] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.852337 7923 raft_consensus.cc:374] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:10.852285 7924 raft_consensus.cc:348] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.852372 7923 raft_consensus.cc:729] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953d7569ba97416d960f3afcdf599679, State: Initialized, Role: FOLLOWER
I20260809 04:40:10.852413 7924 raft_consensus.cc:374] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:10.852454 7924 raft_consensus.cc:729] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153f9577f2f74c18836399fa08d91de5, State: Initialized, Role: FOLLOWER
I20260809 04:40:10.852519 7923 consensus_queue.cc:260] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.852540 7922 raft_consensus.cc:348] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.852633 7922 raft_consensus.cc:374] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:10.852591 7924 consensus_queue.cc:260] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.852674 7922 raft_consensus.cc:729] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da4562a895a474b81211cefb02b4a43, State: Initialized, Role: FOLLOWER
I20260809 04:40:10.852804 7923 ts_tablet_manager.cc:1428] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:10.852797 7922 consensus_queue.cc:260] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:10.853070 7922 ts_tablet_manager.cc:1428] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 04:40:10.853278 7932 tablet_replica.cc:439] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } }
I20260809 04:40:10.853785 7932 tablet_replica.cc:442] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:10.854157 7930 tablet_replica.cc:439] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } }
I20260809 04:40:10.854292 7930 tablet_replica.cc:442] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:10.854442 7929 tablet_replica.cc:439] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } }
I20260809 04:40:10.854554 7929 tablet_replica.cc:442] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:10.854795 7924 ts_tablet_manager.cc:1428] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
W20260809 04:40:10.884517 7895 tablet.cc:2369] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:40:11.002264 7929 raft_consensus.cc:482] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:11.002435 7929 raft_consensus.cc:504] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:11.002938 7929 leader_election.cc:288] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 153f9577f2f74c18836399fa08d91de5 (127.5.222.194:34951), 8da4562a895a474b81211cefb02b4a43 (127.5.222.195:41149)
I20260809 04:40:11.003363 7673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58147cdd2e6641e8b18fea7610498bd6" candidate_uuid: "953d7569ba97416d960f3afcdf599679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153f9577f2f74c18836399fa08d91de5" is_pre_election: true
I20260809 04:40:11.003455 7837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58147cdd2e6641e8b18fea7610498bd6" candidate_uuid: "953d7569ba97416d960f3afcdf599679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da4562a895a474b81211cefb02b4a43" is_pre_election: true
I20260809 04:40:11.003536 7673 raft_consensus.cc:2454] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953d7569ba97416d960f3afcdf599679 in term 0.
I20260809 04:40:11.003630 7837 raft_consensus.cc:2454] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953d7569ba97416d960f3afcdf599679 in term 0.
I20260809 04:40:11.003926 7429 leader_election.cc:302] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 153f9577f2f74c18836399fa08d91de5, 953d7569ba97416d960f3afcdf599679; no voters:
I20260809 04:40:11.004112 7929 raft_consensus.cc:2781] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:11.004171 7929 raft_consensus.cc:482] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:11.004199 7929 raft_consensus.cc:3037] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:11.005247 7929 raft_consensus.cc:504] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:11.005594 7929 leader_election.cc:288] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [CANDIDATE]: Term 1 election: Requested vote from peers 153f9577f2f74c18836399fa08d91de5 (127.5.222.194:34951), 8da4562a895a474b81211cefb02b4a43 (127.5.222.195:41149)
I20260809 04:40:11.006036 7673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58147cdd2e6641e8b18fea7610498bd6" candidate_uuid: "953d7569ba97416d960f3afcdf599679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153f9577f2f74c18836399fa08d91de5"
I20260809 04:40:11.006074 7837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58147cdd2e6641e8b18fea7610498bd6" candidate_uuid: "953d7569ba97416d960f3afcdf599679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da4562a895a474b81211cefb02b4a43"
I20260809 04:40:11.006156 7673 raft_consensus.cc:3037] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:11.006249 7837 raft_consensus.cc:3037] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:11.007138 7673 raft_consensus.cc:2454] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953d7569ba97416d960f3afcdf599679 in term 1.
I20260809 04:40:11.007335 7837 raft_consensus.cc:2454] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953d7569ba97416d960f3afcdf599679 in term 1.
I20260809 04:40:11.007503 7429 leader_election.cc:302] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 153f9577f2f74c18836399fa08d91de5, 953d7569ba97416d960f3afcdf599679; no voters:
I20260809 04:40:11.007812 7929 raft_consensus.cc:2781] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:11.007918 7929 raft_consensus.cc:686] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [term 1 LEADER]: Becoming Leader. State: Replica: 953d7569ba97416d960f3afcdf599679, State: Running, Role: LEADER
I20260809 04:40:11.008104 7929 consensus_queue.cc:237] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } }
I20260809 04:40:11.008884 7957 tablet_replica.cc:439] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: TxnStatusTablet state changed. Reason: New leader 953d7569ba97416d960f3afcdf599679. Latest consensus state: current_term: 1 leader_uuid: "953d7569ba97416d960f3afcdf599679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } }
I20260809 04:40:11.009018 7957 tablet_replica.cc:442] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:40:11.009459 7973 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 04:40:11.009662 7973 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 04:40:11.010043 7322 catalog_manager.cc:5495] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 953d7569ba97416d960f3afcdf599679 (127.5.222.193). New cstate: current_term: 1 leader_uuid: "953d7569ba97416d960f3afcdf599679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } health_report { overall_health: UNKNOWN } } }
I20260809 04:40:11.047278 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:11.043157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51148 (local address 127.5.222.193:34435)
0809 04:40:11.043295 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:11.043299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:11.043428 (+ 129us) server_negotiation.cc:407] Connection header received
0809 04:40:11.043608 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:11.043611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:11.043664 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:11.043747 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:11.044065 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.044556 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.045835 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.046059 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.046134 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:11.046486 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:11.046894 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:11.046986 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:11.047111 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 04:40:11.047162 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:40:11.047343 7968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:11.043062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:34435 (local address 127.0.0.1:51148)
0809 04:40:11.043347 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:11.043363 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:11.043477 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:11.043788 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:11.043791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:11.043800 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:40:11.043927 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:11.043933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.044667 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.044671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:11.045708 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:11.045720 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.046166 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.046169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:11.046270 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:11.046325 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:40:11.047002 (+ 677us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:40:11.047018 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 04:40:11.047093 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 04:40:11.047184 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174}
I20260809 04:40:11.050668 7673 raft_consensus.cc:1261] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5 [term 1 FOLLOWER]: Refusing update from remote peer 953d7569ba97416d960f3afcdf599679: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:11.050776 7837 raft_consensus.cc:1261] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43 [term 1 FOLLOWER]: Refusing update from remote peer 953d7569ba97416d960f3afcdf599679: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:11.051240 7957 consensus_queue.cc:1035] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:11.051455 7929 consensus_queue.cc:1035] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:11.052977 7930 tablet_replica.cc:439] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5: TxnStatusTablet state changed. Reason: New leader 953d7569ba97416d960f3afcdf599679. Latest consensus state: current_term: 1 leader_uuid: "953d7569ba97416d960f3afcdf599679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } }
I20260809 04:40:11.053090 7930 tablet_replica.cc:442] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:11.053936 7932 tablet_replica.cc:439] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43: TxnStatusTablet state changed. Reason: New leader 953d7569ba97416d960f3afcdf599679. Latest consensus state: current_term: 1 leader_uuid: "953d7569ba97416d960f3afcdf599679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } }
I20260809 04:40:11.054026 7932 tablet_replica.cc:442] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:11.055579 7929 tablet_replica.cc:439] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "953d7569ba97416d960f3afcdf599679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } }
I20260809 04:40:11.055685 7929 tablet_replica.cc:442] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:40:11.055781 7939 mvcc.cc:205] Tried to move back new op lower bound from 7316481683658285056 to 7316481683490668544. Current Snapshot: MvccSnapshot[applied={T|T < 7316481683658285056}]
I20260809 04:40:11.056958 7932 tablet_replica.cc:439] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "953d7569ba97416d960f3afcdf599679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } }
I20260809 04:40:11.057066 7932 tablet_replica.cc:442] T 58147cdd2e6641e8b18fea7610498bd6 P 8da4562a895a474b81211cefb02b4a43: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:11.059818 7979 tablet_replica.cc:439] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "953d7569ba97416d960f3afcdf599679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } }
I20260809 04:40:11.059949 7979 tablet_replica.cc:442] T 58147cdd2e6641e8b18fea7610498bd6 P 953d7569ba97416d960f3afcdf599679: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:40:11.061635 7930 tablet_replica.cc:439] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "953d7569ba97416d960f3afcdf599679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953d7569ba97416d960f3afcdf599679" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 34435 } } peers { permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 } } peers { permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 } } }
I20260809 04:40:11.061754 7930 tablet_replica.cc:442] T 58147cdd2e6641e8b18fea7610498bd6 P 153f9577f2f74c18836399fa08d91de5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:11.065183 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:11.060930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51162 (local address 127.5.222.193:34435)
0809 04:40:11.061112 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:11.061117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:11.061128 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:40:11.061180 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:11.061183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:11.061227 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:11.061314 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:11.062040 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.062852 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.063955 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.064309 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.064409 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:11.064700 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:11.064775 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:11.064830 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:11.064980 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 04:40:11.065022 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260809 04:40:11.075942 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:11.072029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:52149 (local address 127.5.222.193:34435)
0809 04:40:11.072163 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:11.072170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:11.072469 (+ 299us) server_negotiation.cc:407] Connection header received
0809 04:40:11.072581 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:11.072583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:11.072620 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:11.072675 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:11.072956 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.073529 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.074806 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.075118 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.075214 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:11.075539 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:11.075599 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:11.075651 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:11.075764 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 04:40:11.075803 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:40:11.076354 7984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:11.072016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:34435 (local address 127.5.222.193:52149)
0809 04:40:11.072381 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:11.072396 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:11.072496 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:11.072712 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:11.072715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:11.072722 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:40:11.072857 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:11.072863 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.073627 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.073631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:11.074697 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:11.074708 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.075259 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.075263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:11.075369 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:11.075419 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:40:11.075677 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:40:11.075683 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:40:11.075749 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 04:40:11.075824 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":145,"threads_started":1}
I20260809 04:40:11.082494 7673 raft_consensus.cc:1261] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [term 1 FOLLOWER]: Refusing update from remote peer 953d7569ba97416d960f3afcdf599679: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:11.082584 7837 raft_consensus.cc:1261] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43 [term 1 FOLLOWER]: Refusing update from remote peer 953d7569ba97416d960f3afcdf599679: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:11.082932 7979 consensus_queue.cc:1035] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "153f9577f2f74c18836399fa08d91de5" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:11.083141 7957 consensus_queue.cc:1035] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da4562a895a474b81211cefb02b4a43" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 41149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:11.125448 7322 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:11.125624 7322 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:11.127709 7322 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebe97641f6c4498ebf064fc61e8f78f2: Sending DeleteTablet for 3 replicas of tablet 8bc75f53e79a45be872a62de301e58e8
I20260809 04:40:11.128286 7484 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bc75f53e79a45be872a62de301e58e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:11 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:51132
I20260809 04:40:11.128288 7651 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bc75f53e79a45be872a62de301e58e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:11 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:50802
I20260809 04:40:11.128492 7948 tablet_replica.cc:330] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679: stopping tablet replica
I20260809 04:40:11.128506 7949 tablet_replica.cc:330] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5: stopping tablet replica
I20260809 04:40:11.128513 7813 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bc75f53e79a45be872a62de301e58e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:11 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:35286
I20260809 04:40:11.128619 7948 raft_consensus.cc:2229] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:11.128641 7949 raft_consensus.cc:2229] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:11.128752 7949 raft_consensus.cc:2258] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:11.128892 7948 raft_consensus.cc:2258] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:11.129488 7947 tablet_replica.cc:330] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43: stopping tablet replica
I20260809 04:40:11.129554 7949 ts_tablet_manager.cc:1905] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:11.129612 7947 raft_consensus.cc:2229] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:11.129733 7947 raft_consensus.cc:2258] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:11.130710 7947 ts_tablet_manager.cc:1905] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:11.132663 7949 ts_tablet_manager.cc:1918] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:11.132748 7949 log.cc:1196] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/8bc75f53e79a45be872a62de301e58e8
I20260809 04:40:11.133019 7949 ts_tablet_manager.cc:1939] T 8bc75f53e79a45be872a62de301e58e8 P 153f9577f2f74c18836399fa08d91de5: Deleting consensus metadata
I20260809 04:40:11.133668 7948 ts_tablet_manager.cc:1905] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:11.133777 7305 catalog_manager.cc:4841] TS 153f9577f2f74c18836399fa08d91de5 (127.5.222.194:34951): tablet 8bc75f53e79a45be872a62de301e58e8 (table test-table [id=5de5051125654301a8fcff0be787a120]) successfully deleted
I20260809 04:40:11.135812 7948 ts_tablet_manager.cc:1918] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:11.135908 7948 log.cc:1196] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-0/wal/wals/8bc75f53e79a45be872a62de301e58e8
I20260809 04:40:11.136145 7948 ts_tablet_manager.cc:1939] T 8bc75f53e79a45be872a62de301e58e8 P 953d7569ba97416d960f3afcdf599679: Deleting consensus metadata
I20260809 04:40:11.136761 7303 catalog_manager.cc:4841] TS 953d7569ba97416d960f3afcdf599679 (127.5.222.193:34435): tablet 8bc75f53e79a45be872a62de301e58e8 (table test-table [id=5de5051125654301a8fcff0be787a120]) successfully deleted
I20260809 04:40:11.137075 7947 ts_tablet_manager.cc:1918] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:11.137197 7947 log.cc:1196] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/8bc75f53e79a45be872a62de301e58e8
I20260809 04:40:11.137490 7947 ts_tablet_manager.cc:1939] T 8bc75f53e79a45be872a62de301e58e8 P 8da4562a895a474b81211cefb02b4a43: Deleting consensus metadata
I20260809 04:40:11.138126 7306 catalog_manager.cc:4841] TS 8da4562a895a474b81211cefb02b4a43 (127.5.222.195:41149): tablet 8bc75f53e79a45be872a62de301e58e8 (table test-table [id=5de5051125654301a8fcff0be787a120]) successfully deleted
Aug 09 04:40:11 dist-test-slave-vxj2 krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250410, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.222.193@KRBTEST.COM
I20260809 04:40:11.148100 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:11.129942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51168 (local address 127.5.222.193:34435)
0809 04:40:11.130052 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:11.130058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:11.138300 (+ 8242us) server_negotiation.cc:407] Connection header received
0809 04:40:11.138509 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:11.138512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:11.138562 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:11.138649 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:11.139347 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.139844 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.140694 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.140909 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.141024 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:11.144432 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:11.144450 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:11.144461 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:11.144484 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:11.146306 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:11.146758 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:11.146764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:11.146766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:11.146849 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:11.147155 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:11.147158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:11.147160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:11.147394 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:11.147634 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:11.147782 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 04:40:11.147900 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 04:40:11.148873 7553 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51168
I20260809 04:40:11.159142 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:11.150784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.5.222.254:43471)
0809 04:40:11.150915 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:11.150920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:11.151120 (+ 200us) server_negotiation.cc:407] Connection header received
0809 04:40:11.151240 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:11.151243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:11.151284 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:11.151348 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:11.152129 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.152628 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.153814 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:11.154039 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:11.154115 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:11.155688 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:11.155691 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:11.155693 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:11.155718 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:11.157304 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:11.157796 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:11.157799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:11.157801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:11.157887 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:11.158172 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:11.158175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:11.158177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:11.158407 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:11.158604 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:11.158835 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 04:40:11.158969 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 04:40:11.161326 7322 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46176
I20260809 04:40:11.162596 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 7407
I20260809 04:40:11.176182 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 7572
I20260809 04:40:11.182649 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 7735
I20260809 04:40:11.190183 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 7281
2026-08-09T04:40:11Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2928 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:11 dist-test-slave-vxj2 krb5kdc[7997](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:11 dist-test-slave-vxj2 krb5kdc[7997](info): set up 2 sockets
Aug 09 04:40:11 dist-test-slave-vxj2 krb5kdc[7997](info): commencing operation
krb5kdc: starting...
W20260809 04:40:12.288336 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:12 dist-test-slave-vxj2 krb5kdc[7997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:40:12.424443 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:41677
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:41677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:12.540755 8011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:12.541038 8011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:12.541079 8011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:12.545218 8011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:12.545279 8011 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 04:40:12.545297 8011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:12.545315 8011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:12.545367 8011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:12.545406 8011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:12.550853 8011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:41677
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:41677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:12.551949 8011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:12.552999 8011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:12.561414 8017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:12.561580 8011 server_base.cc:1034] running on GCE node
W20260809 04:40:12.561450 8019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:12.561460 8016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:12.562219 8011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:12.563345 8011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:12.564544 8011 hybrid_clock.cc:648] HybridClock initialized: now 1786250412564533 us; error 26 us; skew 500 ppm
Aug 09 04:40:12 dist-test-slave-vxj2 krb5kdc[7997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250412, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:12.567271 8011 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:40:12.568121 8011 webserver.cc:460] Webserver started at http://127.5.222.254:38201/ using document root <none> and password file <none>
I20260809 04:40:12.568632 8011 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:12.568691 8011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:12.568848 8011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:12.571044 8011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "e163dcdbf7454fa29f967756cf6590c4"
format_stamp: "Formatted at 2026-08-09 04:40:12 on dist-test-slave-vxj2"
server_key: "8c96953b84cb2c6e6fd0fb239062c3fe"
server_key_iv: "1443478cd6ef070aa34e5e70338f9c34"
server_key_version: "encryptionkey@0"
I20260809 04:40:12.571636 8011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "e163dcdbf7454fa29f967756cf6590c4"
format_stamp: "Formatted at 2026-08-09 04:40:12 on dist-test-slave-vxj2"
server_key: "8c96953b84cb2c6e6fd0fb239062c3fe"
server_key_iv: "1443478cd6ef070aa34e5e70338f9c34"
server_key_version: "encryptionkey@0"
I20260809 04:40:12.575539 8011 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260809 04:40:12.577982 8026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:12.579238 8011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:40:12.579344 8011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "e163dcdbf7454fa29f967756cf6590c4"
format_stamp: "Formatted at 2026-08-09 04:40:12 on dist-test-slave-vxj2"
server_key: "8c96953b84cb2c6e6fd0fb239062c3fe"
server_key_iv: "1443478cd6ef070aa34e5e70338f9c34"
server_key_version: "encryptionkey@0"
I20260809 04:40:12.579449 8011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:12.600967 8011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:12.603935 8011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:12.604092 8011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:12.612483 8011 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:41677
I20260809 04:40:12.612498 8100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:41677 every 8 connection(s)
I20260809 04:40:12.613533 8011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:12.616238 8101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:12.621304 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 8011
I20260809 04:40:12.621402 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:12.621680 6011 external_mini_cluster.cc:1428] Setting key a6bcbf11aee1064445fad109ba48e9d4
I20260809 04:40:12.621970 8101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4: Bootstrap starting.
I20260809 04:40:12.624558 8101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:12.625350 8101 log.cc:824] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:12.627605 8101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4: No bootstrap required, opened a new log
Aug 09 04:40:12 dist-test-slave-vxj2 krb5kdc[7997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:12.631276 8101 raft_consensus.cc:348] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e163dcdbf7454fa29f967756cf6590c4" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41677 } }
I20260809 04:40:12.631520 8101 raft_consensus.cc:374] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:12.631596 8101 raft_consensus.cc:729] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e163dcdbf7454fa29f967756cf6590c4, State: Initialized, Role: FOLLOWER
I20260809 04:40:12.632114 8101 consensus_queue.cc:260] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e163dcdbf7454fa29f967756cf6590c4" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41677 } }
I20260809 04:40:12.632282 8101 raft_consensus.cc:388] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:12.632354 8101 raft_consensus.cc:482] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:12.632521 8101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:12.633502 8101 raft_consensus.cc:504] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e163dcdbf7454fa29f967756cf6590c4" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41677 } }
I20260809 04:40:12.633827 8101 leader_election.cc:302] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e163dcdbf7454fa29f967756cf6590c4; no voters:
I20260809 04:40:12.634182 8101 leader_election.cc:288] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:12.634261 8106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:12.634471 8106 raft_consensus.cc:686] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [term 1 LEADER]: Becoming Leader. State: Replica: e163dcdbf7454fa29f967756cf6590c4, State: Running, Role: LEADER
I20260809 04:40:12.634814 8106 consensus_queue.cc:237] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e163dcdbf7454fa29f967756cf6590c4" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41677 } }
I20260809 04:40:12.635466 8101 sys_catalog.cc:564] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:12.636873 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:12.623044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32824 (local address 127.5.222.254:41677)
0809 04:40:12.623523 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:12.623539 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:12.623575 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:40:12.624196 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:12.624212 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:12.624553 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:12.624902 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:12.625792 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:12.627240 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:12.628122 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:12.628599 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:12.628790 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:12.631383 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:12.631389 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:12.631402 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:12.631434 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:12.633377 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:12.633879 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:12.633884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:12.633889 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:12.634010 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:12.634335 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:12.634340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:12.634364 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:12.634904 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:12.635080 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:12.635473 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 04:40:12.635928 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":162,"threads_started":1}
I20260809 04:40:12.637621 8108 sys_catalog.cc:455] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e163dcdbf7454fa29f967756cf6590c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e163dcdbf7454fa29f967756cf6590c4" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41677 } } }
I20260809 04:40:12.637738 8108 sys_catalog.cc:458] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:12.637971 8109 sys_catalog.cc:455] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e163dcdbf7454fa29f967756cf6590c4. Latest consensus state: current_term: 1 leader_uuid: "e163dcdbf7454fa29f967756cf6590c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e163dcdbf7454fa29f967756cf6590c4" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41677 } } }
I20260809 04:40:12.638270 8109 sys_catalog.cc:458] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4 [sys.catalog]: This master's current role is: LEADER
W20260809 04:40:12.640551 8128 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:40:12.640630 8128 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:40:12.640708 8130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:12.641638 8130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:12.647027 8130 catalog_manager.cc:1347] Generated new cluster ID: 4a9fa098797d4f9a87e864ab939eec91
I20260809 04:40:12.647156 8130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:12.671659 8130 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:12.672957 8130 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:12.681560 8130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4: Generated new TSK 0
I20260809 04:40:12.682365 8130 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:40:12.718501 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41677
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:12.830628 8137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:12.830904 8137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:12.830952 8137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:12.835127 8137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:12.835196 8137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:12.835238 8137 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 04:40:12.835253 8137 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:40:12.835321 8137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:12.840798 8137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.222.254:41677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:12.841928 8137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:12.842954 8137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:12.852396 8143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:12.852396 8142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:12.852465 8147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:12.852790 8137 server_base.cc:1034] running on GCE node
I20260809 04:40:12.853123 8137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:12.853791 8137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:12.854947 8137 hybrid_clock.cc:648] HybridClock initialized: now 1786250412854914 us; error 44 us; skew 500 ppm
Aug 09 04:40:12 dist-test-slave-vxj2 krb5kdc[7997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250412, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:12.857903 8137 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:40:12.858716 8137 webserver.cc:460] Webserver started at http://127.5.222.193:44471/ using document root <none> and password file <none>
I20260809 04:40:12.859262 8137 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:12.859323 8137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:12.859555 8137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:12.861662 8137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "666bc308874544f0bd5b536ca387ad50"
format_stamp: "Formatted at 2026-08-09 04:40:12 on dist-test-slave-vxj2"
server_key: "846125147fa1feca654f92247c5ae28e"
server_key_iv: "292077ac773ae84b0a48e2f3667b66e2"
server_key_version: "encryptionkey@0"
I20260809 04:40:12.862265 8137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "666bc308874544f0bd5b536ca387ad50"
format_stamp: "Formatted at 2026-08-09 04:40:12 on dist-test-slave-vxj2"
server_key: "846125147fa1feca654f92247c5ae28e"
server_key_iv: "292077ac773ae84b0a48e2f3667b66e2"
server_key_version: "encryptionkey@0"
I20260809 04:40:12.866487 8137 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:40:12.868888 8155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:12.870239 8137 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:12.870394 8137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "666bc308874544f0bd5b536ca387ad50"
format_stamp: "Formatted at 2026-08-09 04:40:12 on dist-test-slave-vxj2"
server_key: "846125147fa1feca654f92247c5ae28e"
server_key_iv: "292077ac773ae84b0a48e2f3667b66e2"
server_key_version: "encryptionkey@0"
I20260809 04:40:12.870507 8137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:12.911360 8137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:12.914636 8137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:12.914901 8137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:12.916497 8137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:12.916574 8137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:12.916648 8137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:12.916700 8137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:40:12 dist-test-slave-vxj2 krb5kdc[7997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250412, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:12.928274 8137 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:35549
I20260809 04:40:12.928371 8284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:35549 every 8 connection(s)
I20260809 04:40:12.929488 8137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:12.931854 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:12.917374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:40995 (local address 127.5.222.254:41677)
0809 04:40:12.917531 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:12.917535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:12.918678 (+ 1143us) server_negotiation.cc:407] Connection header received
0809 04:40:12.919796 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:12.919799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:12.919864 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:12.919927 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:12.921743 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:12.922433 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:12.924358 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:12.924680 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:12.924767 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:12.927821 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:12.927825 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:12.927828 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:12.927863 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:12.929649 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:12.930252 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:12.930255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:12.930257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:12.930352 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:12.930744 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:12.930747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:12.930748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:12.930942 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:12.931056 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:12.931549 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 04:40:12.931680 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:40:12.932571 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:12.917981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41677 (local address 127.5.222.193:40995)
0809 04:40:12.918496 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:12.918541 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:12.919548 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:12.920367 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:12.920382 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:12.920871 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:12.921584 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:12.921604 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:12.922606 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:12.922610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:12.924205 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:12.924219 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:12.924838 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:12.924846 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:12.925061 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:12.925697 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:12.925721 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:12.927651 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:12.929761 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:12.929765 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:12.929776 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:12.930129 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:12.930477 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:12.930480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:12.930482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:12.930632 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:12.931068 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:12.931076 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:12.931302 (+ 226us) client_negotiation.cc:769] Sending connection context
0809 04:40:12.931479 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 04:40:12.931791 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":156,"threads_started":1}
I20260809 04:40:12.933902 8287 heartbeater.cc:347] Connected to a master server at 127.5.222.254:41677
I20260809 04:40:12.934075 8287 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:12.934685 8287 heartbeater.cc:511] Master 127.5.222.254:41677 requested a full tablet report, sending...
I20260809 04:40:12.935636 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 8137
I20260809 04:40:12.935786 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:12.936159 6011 external_mini_cluster.cc:1428] Setting key ae4b0f3e558bd4e04f65b80e5670c8a4
I20260809 04:40:12.936507 8047 ts_manager.cc:194] Registered new tserver with Master: 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549)
I20260809 04:40:12.938072 8047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:40995
I20260809 04:40:12.945367 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:12.937749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:37045 (local address 127.5.222.254:41677)
0809 04:40:12.937857 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:12.937863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:12.937879 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:40:12.937995 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:12.938000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:12.938053 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:12.938139 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:12.938638 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:12.939448 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:12.940720 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:12.941103 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:12.941253 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:12.942290 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:12.942294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:12.942296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:12.942319 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:12.943741 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:12.944158 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:12.944162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:12.944164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:12.944245 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:12.944520 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:12.944524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:12.944526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:12.944689 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:12.944798 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:12.945069 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 04:40:12.945216 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:40:12.945380 8295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:12.937417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41677 (local address 127.5.222.193:37045)
0809 04:40:12.937817 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:12.937830 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:12.937905 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:12.938182 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:12.938185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:12.938394 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:12.938512 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:12.938519 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:12.939590 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:12.939595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:12.940576 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:12.940590 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:12.941245 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:12.941247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:12.941317 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:12.941817 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:12.941828 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:12.942147 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:12.943853 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:12.943858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:12.943860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:12.944058 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:12.944336 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:12.944341 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:12.944343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:12.944427 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:12.944790 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:12.944793 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:12.944890 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 04:40:12.945014 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 04:40:12.945191 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:40:12.964427 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41677
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:13.076359 8298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:13.076598 8298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:13.076644 8298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:13.080703 8298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:13.080754 8298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:13.080792 8298 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 04:40:13.080824 8298 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:40:13.080855 8298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:13.086037 8298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.222.254:41677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:13.086962 8298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:13.087864 8298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:40:13.097517 8298 server_base.cc:1034] running on GCE node
W20260809 04:40:13.097481 8305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:13.097468 8304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:13.097446 8307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:13.098120 8298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:13.098711 8298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:13.099879 8298 hybrid_clock.cc:648] HybridClock initialized: now 1786250413099849 us; error 43 us; skew 500 ppm
Aug 09 04:40:13 dist-test-slave-vxj2 krb5kdc[7997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250413, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:13.102778 8298 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:40:13.103668 8298 webserver.cc:460] Webserver started at http://127.5.222.194:45141/ using document root <none> and password file <none>
I20260809 04:40:13.104252 8298 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:13.104341 8298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:13.104573 8298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:13.106905 8298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "1c7931f4b9ad40c899fe1751ae813704"
format_stamp: "Formatted at 2026-08-09 04:40:13 on dist-test-slave-vxj2"
server_key: "c770dde1e65fd66cd6cb45a8091f7501"
server_key_iv: "e82f774b7ec9bb702c04f514d9b260a4"
server_key_version: "encryptionkey@0"
I20260809 04:40:13.107470 8298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "1c7931f4b9ad40c899fe1751ae813704"
format_stamp: "Formatted at 2026-08-09 04:40:13 on dist-test-slave-vxj2"
server_key: "c770dde1e65fd66cd6cb45a8091f7501"
server_key_iv: "e82f774b7ec9bb702c04f514d9b260a4"
server_key_version: "encryptionkey@0"
I20260809 04:40:13.111330 8298 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:40:13.113683 8315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:13.114943 8298 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:13.115087 8298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "1c7931f4b9ad40c899fe1751ae813704"
format_stamp: "Formatted at 2026-08-09 04:40:13 on dist-test-slave-vxj2"
server_key: "c770dde1e65fd66cd6cb45a8091f7501"
server_key_iv: "e82f774b7ec9bb702c04f514d9b260a4"
server_key_version: "encryptionkey@0"
I20260809 04:40:13.115198 8298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:13.162606 8298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:13.165751 8298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:13.165966 8298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:13.167452 8298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:13.167537 8298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:13.167613 8298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:13.167657 8298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:40:13 dist-test-slave-vxj2 krb5kdc[7997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250413, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:13.179281 8298 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:35219
I20260809 04:40:13.179365 8448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:35219 every 8 connection(s)
I20260809 04:40:13.180379 8298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:13.181427 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 8298
I20260809 04:40:13.181543 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:13.181872 6011 external_mini_cluster.cc:1428] Setting key ed5af7cbcc75fc46fce16f8223355f2b
I20260809 04:40:13.183162 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.168848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41677 (local address 127.5.222.194:52577)
0809 04:40:13.169364 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:13.169405 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:13.170215 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:13.170893 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:13.170927 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:13.171315 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:13.171829 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.171843 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.173004 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.173008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.174150 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.174160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.174773 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.174778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.174992 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.175661 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:13.175688 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:13.177744 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:13.179749 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:13.179753 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:13.179766 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:13.180091 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:13.180381 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:13.180384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:13.180388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:13.180593 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:13.181192 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:13.181200 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:13.181531 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 04:40:13.181767 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 04:40:13.182123 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":143,"threads_started":1}
I20260809 04:40:13.184023 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.168291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:52577 (local address 127.5.222.254:41677)
0809 04:40:13.168424 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:13.168429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:13.169525 (+ 1096us) server_negotiation.cc:407] Connection header received
0809 04:40:13.170422 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:13.170425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:13.170483 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:13.170568 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:13.171981 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.172858 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.174278 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.174598 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.174718 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.177944 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:13.177947 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:13.177950 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:13.177980 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:13.179639 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:13.180192 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:13.180195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:13.180196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:13.180290 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:13.180711 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:13.180714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:13.180716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:13.180985 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:13.181104 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:13.183805 (+ 2701us) server_negotiation.cc:299] Negotiation successful
0809 04:40:13.183906 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:40:13.185035 8449 heartbeater.cc:347] Connected to a master server at 127.5.222.254:41677
I20260809 04:40:13.185215 8449 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:13.185774 8449 heartbeater.cc:511] Master 127.5.222.254:41677 requested a full tablet report, sending...
I20260809 04:40:13.186826 8047 ts_manager.cc:194] Registered new tserver with Master: 1c7931f4b9ad40c899fe1751ae813704 (127.5.222.194:35219)
I20260809 04:40:13.187397 8047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:52577
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
I20260809 04:40:13.196892 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.189534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:50879 (local address 127.5.222.254:41677)
0809 04:40:13.189648 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:13.189652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:13.189893 (+ 241us) server_negotiation.cc:407] Connection header received
0809 04:40:13.190036 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:13.190039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:13.190085 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:13.190167 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:13.190607 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.191101 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.191910 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.192143 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.192260 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.193496 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:13.193500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:13.193502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:13.193533 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:13.195062 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:13.195482 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:13.195486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:13.195487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:13.195569 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:13.195861 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:13.195865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:13.195867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:13.196097 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:13.196214 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:13.196621 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 04:40:13.196737 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 04:40:13.196918 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.189506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41677 (local address 127.5.222.194:50879)
0809 04:40:13.189807 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:13.189819 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:13.189931 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:13.190182 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:13.190185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:13.190398 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:13.190491 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.190497 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.191229 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.191232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.191787 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.191795 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.192253 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.192257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.192358 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.192899 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:13.192910 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:13.193348 (+ 438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:13.195170 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:13.195176 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:13.195178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:13.195379 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:13.195671 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:13.195674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:13.195676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:13.195769 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:13.196200 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:13.196204 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:13.196413 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 04:40:13.196547 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 04:40:13.196696 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":88,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:40:13.210410 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41677
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:13.322270 8461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:13.322571 8461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:13.322655 8461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:13.326742 8461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:13.326821 8461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:13.326895 8461 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 04:40:13.326938 8461 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:40:13.327008 8461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:13.332136 8461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.222.254:41677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:13.333137 8461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:13.334257 8461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:13.344223 8469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:13.344249 8466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:13.344370 8461 server_base.cc:1034] running on GCE node
W20260809 04:40:13.344249 8467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:13.344921 8461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:13.345783 8461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:13.347024 8461 hybrid_clock.cc:648] HybridClock initialized: now 1786250413346996 us; error 45 us; skew 500 ppm
Aug 09 04:40:13 dist-test-slave-vxj2 krb5kdc[7997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250413, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:13.349916 8461 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:40:13.350749 8461 webserver.cc:460] Webserver started at http://127.5.222.195:42681/ using document root <none> and password file <none>
I20260809 04:40:13.351306 8461 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:13.351369 8461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:13.351621 8461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:13.353788 8461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "4d4be658b5a54419b49eb76a8220b7a2"
format_stamp: "Formatted at 2026-08-09 04:40:13 on dist-test-slave-vxj2"
server_key: "62e78210c6eab076cf839ec66adcd278"
server_key_iv: "1385cdd7a5e620018d52417ac3590cac"
server_key_version: "encryptionkey@0"
I20260809 04:40:13.354348 8461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "4d4be658b5a54419b49eb76a8220b7a2"
format_stamp: "Formatted at 2026-08-09 04:40:13 on dist-test-slave-vxj2"
server_key: "62e78210c6eab076cf839ec66adcd278"
server_key_iv: "1385cdd7a5e620018d52417ac3590cac"
server_key_version: "encryptionkey@0"
I20260809 04:40:13.358165 8461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:40:13.360780 8480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:13.362304 8461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:40:13.362470 8461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "4d4be658b5a54419b49eb76a8220b7a2"
format_stamp: "Formatted at 2026-08-09 04:40:13 on dist-test-slave-vxj2"
server_key: "62e78210c6eab076cf839ec66adcd278"
server_key_iv: "1385cdd7a5e620018d52417ac3590cac"
server_key_version: "encryptionkey@0"
I20260809 04:40:13.362597 8461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:13.382588 8461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:13.387234 8461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:13.387470 8461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:13.389493 8461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:13.389569 8461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:13.389664 8461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:13.389721 8461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:40:13 dist-test-slave-vxj2 krb5kdc[7997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250413, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:13.401051 8461 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:38095
I20260809 04:40:13.401115 8614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:38095 every 8 connection(s)
I20260809 04:40:13.402213 8461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:13.404232 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.392021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:53115 (local address 127.5.222.254:41677)
0809 04:40:13.392140 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:13.392145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:13.392161 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:40:13.392822 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:13.392827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:13.392898 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:13.393005 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:13.394554 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.395461 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.397095 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.397748 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.397873 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.400367 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:13.400374 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:13.400380 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:13.400416 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:13.402117 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:13.402733 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:13.402736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:13.402738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:13.402829 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:13.403231 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:13.403234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:13.403236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:13.403457 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:13.403542 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:13.404013 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 04:40:13.404115 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:40:13.404956 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.391021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41677 (local address 127.5.222.195:53115)
0809 04:40:13.391464 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:13.391514 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:13.392531 (+ 1017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:13.393217 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:13.393225 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:13.393658 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:13.394376 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.394396 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.395621 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.395625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.396968 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.396982 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.397929 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.397934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.398081 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.398669 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:13.398685 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:13.400191 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:13.402252 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:13.402278 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:13.402296 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:13.402596 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:13.402942 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:13.402946 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:13.402948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:13.403117 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:13.403557 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:13.403562 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:13.403810 (+ 248us) client_negotiation.cc:769] Sending connection context
0809 04:40:13.403957 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 04:40:13.404181 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":137,"threads_started":1}
I20260809 04:40:13.406184 8615 heartbeater.cc:347] Connected to a master server at 127.5.222.254:41677
I20260809 04:40:13.406338 8615 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:13.406826 8615 heartbeater.cc:511] Master 127.5.222.254:41677 requested a full tablet report, sending...
I20260809 04:40:13.407517 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 8461
I20260809 04:40:13.407642 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:40:13.407966 6011 external_mini_cluster.cc:1428] Setting key 48cda83aecc09a5ce5a9b4ec40f6f852
I20260809 04:40:13.408496 8048 ts_manager.cc:194] Registered new tserver with Master: 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095)
I20260809 04:40:13.409343 8048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:53115
I20260809 04:40:13.410152 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:40:13.417052 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.408564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:40673 (local address 127.5.222.254:41677)
0809 04:40:13.408678 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:13.408682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:13.409043 (+ 361us) server_negotiation.cc:407] Connection header received
0809 04:40:13.409295 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:13.409298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:13.409344 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:13.409425 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:13.409899 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.410588 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.411638 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.412036 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.412155 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.413383 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:13.413386 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:13.413388 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:13.413409 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:13.415221 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:13.415685 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:13.415689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:13.415690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:13.415797 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:13.416108 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:13.416112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:13.416114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:13.416362 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:13.416486 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:13.416765 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 04:40:13.416897 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 04:40:13.417062 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.408491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41677 (local address 127.5.222.195:40673)
0809 04:40:13.408947 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:13.408962 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:13.409203 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:13.409464 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:13.409467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:13.409685 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:13.409786 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.409792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.410732 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.410735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.411514 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.411522 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.412170 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.412176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.412294 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.412758 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:13.412777 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:13.413251 (+ 474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:13.415354 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:13.415357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:13.415359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:13.415579 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:13.415904 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:13.415907 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:13.415908 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:13.415994 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:13.416474 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:13.416477 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:13.416577 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 04:40:13.416690 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 04:40:13.416857 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":301,"threads_started":1}
Aug 09 04:40:13 dist-test-slave-vxj2 krb5kdc[7997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:40:13 dist-test-slave-vxj2 krb5kdc[7997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:13.439270 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.430293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.5.222.254:41677)
0809 04:40:13.430416 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:13.430420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:13.430431 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:40:13.430467 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:13.430469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:13.430505 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:13.430595 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:13.431495 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.432011 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.432869 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.433085 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.433232 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.435882 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:13.435887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:13.435889 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:13.435917 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:13.437538 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:13.438096 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:13.438098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:13.438099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:13.438176 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:13.438455 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:13.438457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:13.438458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:13.438620 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:13.438717 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:13.438999 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 04:40:13.439093 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:40:13.442289 8047 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:13.445452 8047 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:13.459969 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.454112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37334 (local address 127.5.222.193:35549)
0809 04:40:13.454921 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:13.454927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:13.454951 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:40:13.455027 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:13.455032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:13.455266 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:13.455419 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:13.455775 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.456901 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.458593 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.459272 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.459396 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.459604 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:13.459715 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 04:40:13.459805 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":87,"threads_started":1}
I20260809 04:40:13.459966 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.453973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:35549 (local address 127.0.0.1:37334)
0809 04:40:13.454465 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:13.454514 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:13.454644 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:13.455439 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:13.455445 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:13.455470 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:13.455640 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.455648 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.457038 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.457042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.458459 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.458471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.459407 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.459411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.459531 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.459568 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 04:40:13.459687 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 04:40:13.459804 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260809 04:40:13.460517 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.454653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:35219 (local address 127.0.0.1:52946)
0809 04:40:13.455465 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:13.455477 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:13.455562 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:13.456274 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:13.456280 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:13.456302 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:13.456379 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.456385 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.457672 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.457676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.458830 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.458839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.460123 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.460128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.460227 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.460248 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:40:13.460314 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 04:40:13.460377 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"mutex_wait_us":41,"thread_start_us":111,"threads_started":1}
I20260809 04:40:13.460877 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.454110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:38095 (local address 127.0.0.1:52624)
0809 04:40:13.455180 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:13.455195 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:13.455271 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:13.456088 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:13.456091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:13.456109 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:13.456237 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.456244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.457412 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.457415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.458797 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.458808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.460466 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.460469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.460575 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.460604 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 04:40:13.460681 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 04:40:13.460748 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"mutex_wait_us":70,"thread_start_us":94,"threads_started":1}
I20260809 04:40:13.461062 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.454823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.5.222.194:35219)
0809 04:40:13.455210 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:13.455224 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:13.455801 (+ 577us) server_negotiation.cc:407] Connection header received
0809 04:40:13.455883 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:13.455888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:13.456076 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:13.456222 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:13.456495 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.457545 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.458989 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.459849 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.459939 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.460047 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:13.460811 (+ 764us) server_negotiation.cc:299] Negotiation successful
0809 04:40:13.460899 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":102,"threads_started":1}
I20260809 04:40:13.461400 8640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.454768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.5.222.195:38095)
0809 04:40:13.455741 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:13.455751 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:13.455769 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:40:13.455826 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:13.455829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:13.455952 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:13.456109 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:13.456366 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.457279 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.458946 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.459786 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.459913 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.460406 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:13.461101 (+ 695us) server_negotiation.cc:299] Negotiation successful
0809 04:40:13.461232 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":115,"threads_started":1}
I20260809 04:40:13.463083 8210 tablet_service.cc:1467] Processing CreateTablet for tablet 23f3404bf34a409abacff43d31066ecb (DEFAULT_TABLE table=test-table [id=71eb59a6a5ae47a7a5572c6b5f5a772b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:13.463600 8542 tablet_service.cc:1467] Processing CreateTablet for tablet 23f3404bf34a409abacff43d31066ecb (DEFAULT_TABLE table=test-table [id=71eb59a6a5ae47a7a5572c6b5f5a772b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:13.463615 8372 tablet_service.cc:1467] Processing CreateTablet for tablet 23f3404bf34a409abacff43d31066ecb (DEFAULT_TABLE table=test-table [id=71eb59a6a5ae47a7a5572c6b5f5a772b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:13.464253 8210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23f3404bf34a409abacff43d31066ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:13.464531 8542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23f3404bf34a409abacff43d31066ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:13.464542 8372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23f3404bf34a409abacff43d31066ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:13.471539 8642 tablet_bootstrap.cc:492] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2: Bootstrap starting.
I20260809 04:40:13.472594 8643 tablet_bootstrap.cc:492] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50: Bootstrap starting.
I20260809 04:40:13.472895 8644 tablet_bootstrap.cc:492] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704: Bootstrap starting.
I20260809 04:40:13.474078 8642 tablet_bootstrap.cc:654] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:13.474848 8644 tablet_bootstrap.cc:654] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:13.474874 8643 tablet_bootstrap.cc:654] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:13.474998 8642 log.cc:824] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:13.475692 8644 log.cc:824] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:13.475679 8643 log.cc:824] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:13.477555 8642 tablet_bootstrap.cc:492] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2: No bootstrap required, opened a new log
I20260809 04:40:13.477766 8642 ts_tablet_manager.cc:1397] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:40:13.477979 8643 tablet_bootstrap.cc:492] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50: No bootstrap required, opened a new log
I20260809 04:40:13.477979 8644 tablet_bootstrap.cc:492] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704: No bootstrap required, opened a new log
I20260809 04:40:13.478168 8643 ts_tablet_manager.cc:1397] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:40:13.478168 8644 ts_tablet_manager.cc:1397] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:40:13.480729 8642 raft_consensus.cc:348] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } }
I20260809 04:40:13.480981 8642 raft_consensus.cc:374] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:13.481034 8642 raft_consensus.cc:729] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4be658b5a54419b49eb76a8220b7a2, State: Initialized, Role: FOLLOWER
I20260809 04:40:13.481487 8642 consensus_queue.cc:260] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } }
I20260809 04:40:13.482188 8615 heartbeater.cc:503] Master 127.5.222.254:41677 was elected leader, sending a full tablet report...
I20260809 04:40:13.482059 8644 raft_consensus.cc:348] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } }
I20260809 04:40:13.482307 8644 raft_consensus.cc:374] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:13.482359 8644 raft_consensus.cc:729] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7931f4b9ad40c899fe1751ae813704, State: Initialized, Role: FOLLOWER
I20260809 04:40:13.482304 8643 raft_consensus.cc:348] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } }
I20260809 04:40:13.482537 8643 raft_consensus.cc:374] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:13.482570 8642 ts_tablet_manager.cc:1428] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 04:40:13.482642 8643 raft_consensus.cc:729] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666bc308874544f0bd5b536ca387ad50, State: Initialized, Role: FOLLOWER
I20260809 04:40:13.482777 8644 consensus_queue.cc:260] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } }
I20260809 04:40:13.483098 8643 consensus_queue.cc:260] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } }
I20260809 04:40:13.483500 8449 heartbeater.cc:503] Master 127.5.222.254:41677 was elected leader, sending a full tablet report...
I20260809 04:40:13.483779 8644 ts_tablet_manager.cc:1428] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:40:13.483875 8643 ts_tablet_manager.cc:1428] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:40:13.484160 8287 heartbeater.cc:503] Master 127.5.222.254:41677 was elected leader, sending a full tablet report...
I20260809 04:40:13.629102 8650 raft_consensus.cc:482] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:13.629333 8650 raft_consensus.cc:504] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } }
I20260809 04:40:13.630398 8650 leader_election.cc:288] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549), 1c7931f4b9ad40c899fe1751ae813704 (127.5.222.194:35219)
I20260809 04:40:13.634829 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.630856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:40561 (local address 127.5.222.193:35549)
0809 04:40:13.631013 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:13.631018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:13.631033 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:40:13.631205 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:13.631208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:13.631269 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:13.631362 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:13.631687 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.632339 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.633573 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.634221 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.634371 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.634431 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:13.634606 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 04:40:13.634693 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:40:13.634832 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.630696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:35549 (local address 127.5.222.195:40561)
0809 04:40:13.630909 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:13.630929 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:13.631047 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:13.631393 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:13.631395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:13.631412 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:13.631553 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.631561 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.632454 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.632457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.633458 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.633467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.634397 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.634402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.634501 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.634521 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:40:13.634592 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 04:40:13.634656 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 04:40:13.635550 8231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23f3404bf34a409abacff43d31066ecb" candidate_uuid: "4d4be658b5a54419b49eb76a8220b7a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666bc308874544f0bd5b536ca387ad50" is_pre_election: true
I20260809 04:40:13.635934 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.631172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:58579 (local address 127.5.222.194:35219)
0809 04:40:13.631317 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:13.631321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:13.631520 (+ 199us) server_negotiation.cc:407] Connection header received
0809 04:40:13.631609 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:13.631613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:13.631669 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:13.631742 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:13.632015 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.632582 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.634741 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.635371 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.635488 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.635529 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:13.635721 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 04:40:13.635825 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:40:13.636003 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.630993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:35219 (local address 127.5.222.195:58579)
0809 04:40:13.631403 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:13.631432 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:13.631525 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:13.631764 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:13.631766 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:13.631783 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:13.631903 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.631910 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.632693 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.632696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.634608 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.634618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.635504 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.635507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.635574 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.635592 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 04:40:13.635674 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 04:40:13.635829 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":133,"threads_started":1}
I20260809 04:40:13.636008 8231 raft_consensus.cc:2454] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4be658b5a54419b49eb76a8220b7a2 in term 0.
I20260809 04:40:13.636569 8485 leader_election.cc:302] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d4be658b5a54419b49eb76a8220b7a2, 666bc308874544f0bd5b536ca387ad50; no voters:
I20260809 04:40:13.636804 8394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23f3404bf34a409abacff43d31066ecb" candidate_uuid: "4d4be658b5a54419b49eb76a8220b7a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7931f4b9ad40c899fe1751ae813704" is_pre_election: true
I20260809 04:40:13.636921 8650 raft_consensus.cc:2781] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:13.637025 8650 raft_consensus.cc:482] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:13.637080 8650 raft_consensus.cc:3037] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:13.637104 8394 raft_consensus.cc:2454] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4be658b5a54419b49eb76a8220b7a2 in term 0.
I20260809 04:40:13.638335 8650 raft_consensus.cc:504] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } }
I20260809 04:40:13.638676 8650 leader_election.cc:288] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [CANDIDATE]: Term 1 election: Requested vote from peers 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549), 1c7931f4b9ad40c899fe1751ae813704 (127.5.222.194:35219)
I20260809 04:40:13.639129 8231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23f3404bf34a409abacff43d31066ecb" candidate_uuid: "4d4be658b5a54419b49eb76a8220b7a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666bc308874544f0bd5b536ca387ad50"
I20260809 04:40:13.639201 8394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23f3404bf34a409abacff43d31066ecb" candidate_uuid: "4d4be658b5a54419b49eb76a8220b7a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7931f4b9ad40c899fe1751ae813704"
I20260809 04:40:13.639258 8231 raft_consensus.cc:3037] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:13.639321 8394 raft_consensus.cc:3037] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:13.640640 8231 raft_consensus.cc:2454] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4be658b5a54419b49eb76a8220b7a2 in term 1.
I20260809 04:40:13.640640 8394 raft_consensus.cc:2454] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4be658b5a54419b49eb76a8220b7a2 in term 1.
I20260809 04:40:13.640938 8485 leader_election.cc:302] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7931f4b9ad40c899fe1751ae813704, 4d4be658b5a54419b49eb76a8220b7a2; no voters:
I20260809 04:40:13.641130 8650 raft_consensus.cc:2781] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:13.641433 8650 raft_consensus.cc:686] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 LEADER]: Becoming Leader. State: Replica: 4d4be658b5a54419b49eb76a8220b7a2, State: Running, Role: LEADER
I20260809 04:40:13.641785 8650 consensus_queue.cc:237] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } }
I20260809 04:40:13.644894 8048 catalog_manager.cc:5495] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195). New cstate: current_term: 1 leader_uuid: "4d4be658b5a54419b49eb76a8220b7a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } health_report { overall_health: UNKNOWN } } }
W20260809 04:40:13.653110 8616 tablet.cc:2369] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:40:13.681303 8288 tablet.cc:2369] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:40:13.681552 8451 tablet.cc:2369] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 04:40:13 dist-test-slave-vxj2 krb5kdc[7997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250412, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:13.715608 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.707128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32838 (local address 127.5.222.254:41677)
0809 04:40:13.707318 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:13.707325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:13.707497 (+ 172us) server_negotiation.cc:407] Connection header received
0809 04:40:13.707657 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:13.707660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:13.707701 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:13.707800 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:13.708472 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.709051 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.709872 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.710101 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.710209 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.712476 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:13.712479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:13.712481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:13.712504 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:13.713981 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:13.714466 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:13.714470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:13.714471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:13.714552 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:13.714817 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:13.714820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:13.714821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:13.714989 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:13.715067 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:13.715359 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 04:40:13.715460 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 04:40:13.715615 8665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:13.707009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41677 (local address 127.0.0.1:32838)
0809 04:40:13.707403 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:13.707416 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:13.707558 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:13.707817 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:13.707820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:13.708253 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:13.708354 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.708361 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.709193 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.709196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.709753 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:13.709761 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:13.710210 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:13.710213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:13.710304 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:13.710866 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:13.710878 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:13.712327 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:13.714095 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:13.714104 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:13.714109 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:13.714344 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:13.714643 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:13.714645 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:13.714647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:13.714727 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:13.715091 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:13.715097 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:13.715201 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 04:40:13.715295 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 04:40:13.715408 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":110,"threads_started":1}
I20260809 04:40:13.721240 8048 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:32838:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 04:40:13.722061 8048 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:13.727277 8372 tablet_service.cc:1467] Processing CreateTablet for tablet dddc1e3db2174bae98dfbb48d313e0b3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=35006325961447e89314719db8a33b69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:40:13.727301 8210 tablet_service.cc:1467] Processing CreateTablet for tablet dddc1e3db2174bae98dfbb48d313e0b3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=35006325961447e89314719db8a33b69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:40:13.727479 8542 tablet_service.cc:1467] Processing CreateTablet for tablet dddc1e3db2174bae98dfbb48d313e0b3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=35006325961447e89314719db8a33b69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:40:13.728089 8372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dddc1e3db2174bae98dfbb48d313e0b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:13.728108 8210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dddc1e3db2174bae98dfbb48d313e0b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:13.728163 8542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dddc1e3db2174bae98dfbb48d313e0b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:13.730695 8644 tablet_bootstrap.cc:492] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: Bootstrap starting.
I20260809 04:40:13.731488 8643 tablet_bootstrap.cc:492] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50: Bootstrap starting.
I20260809 04:40:13.731626 8644 tablet_bootstrap.cc:654] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:13.732537 8643 tablet_bootstrap.cc:654] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:13.733000 8644 tablet_bootstrap.cc:492] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: No bootstrap required, opened a new log
I20260809 04:40:13.733105 8644 ts_tablet_manager.cc:1397] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:13.733258 8642 tablet_bootstrap.cc:492] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2: Bootstrap starting.
I20260809 04:40:13.733600 8644 raft_consensus.cc:348] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:13.733783 8644 raft_consensus.cc:374] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:13.733827 8644 raft_consensus.cc:729] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7931f4b9ad40c899fe1751ae813704, State: Initialized, Role: FOLLOWER
I20260809 04:40:13.733954 8644 consensus_queue.cc:260] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:13.734284 8642 tablet_bootstrap.cc:654] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:13.734232 8651 tablet_replica.cc:439] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } }
I20260809 04:40:13.734373 8651 tablet_replica.cc:442] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:13.734665 8644 ts_tablet_manager.cc:1428] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:13.735405 8643 tablet_bootstrap.cc:492] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50: No bootstrap required, opened a new log
I20260809 04:40:13.735512 8643 ts_tablet_manager.cc:1397] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 04:40:13.736034 8643 raft_consensus.cc:348] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:13.736167 8643 raft_consensus.cc:374] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:13.736217 8643 raft_consensus.cc:729] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666bc308874544f0bd5b536ca387ad50, State: Initialized, Role: FOLLOWER
I20260809 04:40:13.736335 8643 consensus_queue.cc:260] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:13.736595 8643 ts_tablet_manager.cc:1428] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 04:40:13.736615 8652 tablet_replica.cc:439] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } }
I20260809 04:40:13.736785 8652 tablet_replica.cc:442] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:13.737598 8642 tablet_bootstrap.cc:492] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2: No bootstrap required, opened a new log
I20260809 04:40:13.737700 8642 ts_tablet_manager.cc:1397] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 04:40:13.738161 8642 raft_consensus.cc:348] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:13.738246 8642 raft_consensus.cc:374] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:13.738266 8642 raft_consensus.cc:729] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4be658b5a54419b49eb76a8220b7a2, State: Initialized, Role: FOLLOWER
I20260809 04:40:13.738430 8642 consensus_queue.cc:260] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:13.738715 8642 ts_tablet_manager.cc:1428] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 04:40:13.738673 8650 tablet_replica.cc:439] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } }
I20260809 04:40:13.738785 8650 tablet_replica.cc:442] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:14.051671 8650 consensus_queue.cc:1035] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:14.057361 8655 consensus_queue.cc:1035] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:14.066845 8651 raft_consensus.cc:482] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:14.066977 8651 raft_consensus.cc:504] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:14.068041 8651 leader_election.cc:288] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549), 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095)
I20260809 04:40:14.072245 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.068306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:44443 (local address 127.5.222.193:35549)
0809 04:40:14.068439 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:14.068444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:14.068554 (+ 110us) server_negotiation.cc:407] Connection header received
0809 04:40:14.068741 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:14.068744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:14.068795 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:14.068873 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:14.069194 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.069927 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.071171 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.071694 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.071782 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.071819 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:14.072053 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 04:40:14.072106 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 04:40:14.072285 8675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.068195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:35549 (local address 127.5.222.194:44443)
0809 04:40:14.068473 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:14.068489 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:14.068610 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:14.068902 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:14.068905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:14.068927 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:14.069051 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:14.069058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.070033 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.070037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:14.071044 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:14.071053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.071823 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.071829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:14.071923 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.071949 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:40:14.072030 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 04:40:14.072100 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":73,"threads_started":1}
I20260809 04:40:14.072647 8676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.068302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:38095 (local address 127.5.222.194:57343)
0809 04:40:14.068635 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:14.068646 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:14.068755 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:14.069011 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:14.069014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:14.069028 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:14.069614 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:14.069625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.070566 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.070569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:14.071414 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:14.071433 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.072202 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.072206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:14.072309 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.072336 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:40:14.072426 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 04:40:14.072509 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":80,"threads_started":1}
I20260809 04:40:14.072778 8231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dddc1e3db2174bae98dfbb48d313e0b3" candidate_uuid: "1c7931f4b9ad40c899fe1751ae813704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666bc308874544f0bd5b536ca387ad50" is_pre_election: true
I20260809 04:40:14.072881 8677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.068456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:57343 (local address 127.5.222.195:38095)
0809 04:40:14.068708 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:14.068713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:14.068771 (+ 58us) server_negotiation.cc:407] Connection header received
0809 04:40:14.068824 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:14.068826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:14.068893 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:14.068978 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:14.069774 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.070453 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.071541 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.072091 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.072198 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.072240 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:14.072634 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 04:40:14.072716 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260809 04:40:14.072929 8231 raft_consensus.cc:2454] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c7931f4b9ad40c899fe1751ae813704 in term 0.
I20260809 04:40:14.073284 8321 leader_election.cc:302] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7931f4b9ad40c899fe1751ae813704, 666bc308874544f0bd5b536ca387ad50; no voters:
I20260809 04:40:14.073364 8562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dddc1e3db2174bae98dfbb48d313e0b3" candidate_uuid: "1c7931f4b9ad40c899fe1751ae813704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4be658b5a54419b49eb76a8220b7a2" is_pre_election: true
I20260809 04:40:14.073558 8562 raft_consensus.cc:2454] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c7931f4b9ad40c899fe1751ae813704 in term 0.
I20260809 04:40:14.073602 8651 raft_consensus.cc:2781] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:14.073653 8651 raft_consensus.cc:482] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:14.073758 8651 raft_consensus.cc:3037] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:14.074839 8651 raft_consensus.cc:504] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:14.075217 8651 leader_election.cc:288] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [CANDIDATE]: Term 1 election: Requested vote from peers 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549), 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095)
I20260809 04:40:14.075493 8231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dddc1e3db2174bae98dfbb48d313e0b3" candidate_uuid: "1c7931f4b9ad40c899fe1751ae813704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666bc308874544f0bd5b536ca387ad50"
I20260809 04:40:14.075604 8231 raft_consensus.cc:3037] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:14.075594 8562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dddc1e3db2174bae98dfbb48d313e0b3" candidate_uuid: "1c7931f4b9ad40c899fe1751ae813704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4be658b5a54419b49eb76a8220b7a2"
I20260809 04:40:14.075735 8562 raft_consensus.cc:3037] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:14.076485 8231 raft_consensus.cc:2454] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c7931f4b9ad40c899fe1751ae813704 in term 1.
I20260809 04:40:14.076701 8562 raft_consensus.cc:2454] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c7931f4b9ad40c899fe1751ae813704 in term 1.
I20260809 04:40:14.076740 8321 leader_election.cc:302] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7931f4b9ad40c899fe1751ae813704, 666bc308874544f0bd5b536ca387ad50; no voters:
I20260809 04:40:14.076897 8651 raft_consensus.cc:2781] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:14.077124 8651 raft_consensus.cc:686] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [term 1 LEADER]: Becoming Leader. State: Replica: 1c7931f4b9ad40c899fe1751ae813704, State: Running, Role: LEADER
I20260809 04:40:14.077458 8651 consensus_queue.cc:237] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:14.078583 8678 tablet_replica.cc:439] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: TxnStatusTablet state changed. Reason: New leader 1c7931f4b9ad40c899fe1751ae813704. Latest consensus state: current_term: 1 leader_uuid: "1c7931f4b9ad40c899fe1751ae813704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } }
I20260809 04:40:14.078704 8678 tablet_replica.cc:442] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:40:14.078998 8680 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 04:40:14.079232 8680 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 04:40:14.079774 8048 catalog_manager.cc:5495] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c7931f4b9ad40c899fe1751ae813704 (127.5.222.194). New cstate: current_term: 1 leader_uuid: "1c7931f4b9ad40c899fe1751ae813704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20260809 04:40:14.100952 8665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.097337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:35219 (local address 127.0.0.1:52962)
0809 04:40:14.097508 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:14.097525 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:14.097645 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:14.097975 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:14.097978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:14.097985 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:40:14.098092 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:14.098098 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.098814 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.098818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:14.099430 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:14.099439 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.099889 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.099892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:14.099983 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.100033 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:40:14.100661 (+ 628us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:40:14.100679 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 04:40:14.100749 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 04:40:14.100836 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 04:40:14.100952 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.097407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52962 (local address 127.5.222.194:35219)
0809 04:40:14.097511 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:14.097517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:14.097590 (+ 73us) server_negotiation.cc:407] Connection header received
0809 04:40:14.097782 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:14.097792 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:14.097846 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:14.097961 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:14.098201 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.098702 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.099551 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.099768 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.099884 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.100177 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:14.100556 (+ 379us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:14.100685 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:14.100758 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 04:40:14.100808 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:40:14.105679 8231 raft_consensus.cc:1261] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Refusing update from remote peer 1c7931f4b9ad40c899fe1751ae813704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:14.105854 8562 raft_consensus.cc:1261] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Refusing update from remote peer 1c7931f4b9ad40c899fe1751ae813704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:14.106132 8678 consensus_queue.cc:1035] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:14.106386 8651 consensus_queue.cc:1035] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:14.109477 8652 tablet_replica.cc:439] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50: TxnStatusTablet state changed. Reason: New leader 1c7931f4b9ad40c899fe1751ae813704. Latest consensus state: current_term: 1 leader_uuid: "1c7931f4b9ad40c899fe1751ae813704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } }
I20260809 04:40:14.109592 8652 tablet_replica.cc:442] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:14.110895 8655 tablet_replica.cc:439] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2: TxnStatusTablet state changed. Reason: New leader 1c7931f4b9ad40c899fe1751ae813704. Latest consensus state: current_term: 1 leader_uuid: "1c7931f4b9ad40c899fe1751ae813704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } }
I20260809 04:40:14.111058 8655 tablet_replica.cc:442] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:14.111881 8678 tablet_replica.cc:439] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c7931f4b9ad40c899fe1751ae813704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } }
I20260809 04:40:14.112003 8678 tablet_replica.cc:442] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:40:14.112299 8678 tablet_replica.cc:439] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c7931f4b9ad40c899fe1751ae813704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } }
I20260809 04:40:14.112380 8678 tablet_replica.cc:442] T dddc1e3db2174bae98dfbb48d313e0b3 P 1c7931f4b9ad40c899fe1751ae813704: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:40:14.112803 8652 tablet_replica.cc:439] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c7931f4b9ad40c899fe1751ae813704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } }
I20260809 04:40:14.112926 8652 tablet_replica.cc:442] T dddc1e3db2174bae98dfbb48d313e0b3 P 666bc308874544f0bd5b536ca387ad50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:14.113708 8674 mvcc.cc:205] Tried to move back new op lower bound from 7316481696170967040 to 7316481696063270912. Current Snapshot: MvccSnapshot[applied={T|T < 7316481696170967040}]
I20260809 04:40:14.114634 8655 tablet_replica.cc:439] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c7931f4b9ad40c899fe1751ae813704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } } }
I20260809 04:40:14.114748 8655 tablet_replica.cc:442] T dddc1e3db2174bae98dfbb48d313e0b3 P 4d4be658b5a54419b49eb76a8220b7a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:14.115496 8670 mvcc.cc:205] Tried to move back new op lower bound from 7316481696170967040 to 7316481696063270912. Current Snapshot: MvccSnapshot[applied={T|T < 7316481696170967040}]
I20260809 04:40:14.115645 8672 mvcc.cc:205] Tried to move back new op lower bound from 7316481696170967040 to 7316481696063270912. Current Snapshot: MvccSnapshot[applied={T|T < 7316481696170967040}]
I20260809 04:40:14.126426 8677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.122542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52638 (local address 127.5.222.195:38095)
0809 04:40:14.122662 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:14.122666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:14.122923 (+ 257us) server_negotiation.cc:407] Connection header received
0809 04:40:14.123112 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:14.123115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:14.123166 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:14.123276 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:14.123541 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.124065 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.124959 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.125244 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.125354 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.125624 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:14.126033 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:14.126160 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:14.126227 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 04:40:14.126283 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 04:40:14.137995 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.134059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:59941 (local address 127.5.222.194:35219)
0809 04:40:14.134195 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:14.134200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:14.134379 (+ 179us) server_negotiation.cc:407] Connection header received
0809 04:40:14.134484 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:14.134487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:14.134527 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:14.134590 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:14.134881 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.135487 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.136882 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.137191 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.137316 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.137615 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:14.137676 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:14.137732 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:14.137849 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 04:40:14.137891 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:40:14.138073 8696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.133978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:35219 (local address 127.5.222.195:59941)
0809 04:40:14.134291 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:14.134306 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:14.134404 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:14.134620 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:14.134623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:14.134631 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:40:14.134745 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:14.134751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.135595 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.135598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:14.136769 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:14.136780 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.137324 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.137328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:14.137423 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.137472 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:40:14.137759 (+ 287us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:40:14.137766 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 04:40:14.137831 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 04:40:14.137909 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":136,"threads_started":1}
I20260809 04:40:14.163480 8677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.159864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:38373 (local address 127.5.222.195:38095)
0809 04:40:14.159987 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:14.159993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:14.160044 (+ 51us) server_negotiation.cc:407] Connection header received
0809 04:40:14.160174 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:14.160178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:14.160228 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:14.160331 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:14.160615 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.161433 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.162326 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.162675 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.162778 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.163056 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:14.163134 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:14.163205 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:14.163288 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 04:40:14.163341 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:40:14.163897 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.159598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:38095 (local address 127.5.222.194:38373)
0809 04:40:14.159955 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:14.159972 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:14.160057 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:14.160344 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:14.160347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:14.160355 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:40:14.160487 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:14.160495 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.161531 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.161534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:14.162203 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:14.162212 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.162818 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.162821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:14.162892 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.162934 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:40:14.163214 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:40:14.163219 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:40:14.163270 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 04:40:14.163371 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":103,"threads_started":1}
I20260809 04:40:14.191558 8047 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:14.191725 8047 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:14.194327 8047 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4: Sending DeleteTablet for 3 replicas of tablet 23f3404bf34a409abacff43d31066ecb
I20260809 04:40:14.195195 8210 tablet_service.cc:1514] Processing DeleteTablet for tablet 23f3404bf34a409abacff43d31066ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:14 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:37334
I20260809 04:40:14.195967 8542 tablet_service.cc:1514] Processing DeleteTablet for tablet 23f3404bf34a409abacff43d31066ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:14 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:52624
I20260809 04:40:14.196520 8372 tablet_service.cc:1514] Processing DeleteTablet for tablet 23f3404bf34a409abacff43d31066ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:14 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:52946
I20260809 04:40:14.197530 8047 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:14.197906 8047 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:14.198117 8701 tablet_replica.cc:330] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50: stopping tablet replica
I20260809 04:40:14.198431 8701 raft_consensus.cc:2229] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:14.198653 8701 raft_consensus.cc:2258] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:14.199993 8701 ts_tablet_manager.cc:1905] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:14.203307 8701 ts_tablet_manager.cc:1918] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:14.203406 8701 log.cc:1196] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/wal/wals/23f3404bf34a409abacff43d31066ecb
I20260809 04:40:14.203351 8542 tablet_service.cc:1467] Processing CreateTablet for tablet 9941dd00900c434baaba7b644880aeab (DEFAULT_TABLE table=test-table [id=10c753a558214e2194932b5b01d8d4f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:14.203641 8542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9941dd00900c434baaba7b644880aeab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:14.203727 8701 ts_tablet_manager.cc:1939] T 23f3404bf34a409abacff43d31066ecb P 666bc308874544f0bd5b536ca387ad50: Deleting consensus metadata
I20260809 04:40:14.205210 8210 tablet_service.cc:1467] Processing CreateTablet for tablet 9941dd00900c434baaba7b644880aeab (DEFAULT_TABLE table=test-table [id=10c753a558214e2194932b5b01d8d4f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:14.205428 8210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9941dd00900c434baaba7b644880aeab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:14.205897 8031 catalog_manager.cc:4841] TS 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549): tablet 23f3404bf34a409abacff43d31066ecb (table test-table [id=71eb59a6a5ae47a7a5572c6b5f5a772b]) successfully deleted
I20260809 04:40:14.206457 8642 tablet_bootstrap.cc:492] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2: Bootstrap starting.
I20260809 04:40:14.207475 8642 tablet_bootstrap.cc:654] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:14.208148 8643 tablet_bootstrap.cc:492] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50: Bootstrap starting.
I20260809 04:40:14.208652 8702 tablet_replica.cc:330] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2: stopping tablet replica
I20260809 04:40:14.208997 8702 raft_consensus.cc:2229] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:14.209055 8643 tablet_bootstrap.cc:654] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:14.209306 8703 tablet_replica.cc:330] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704: stopping tablet replica
I20260809 04:40:14.209795 8702 raft_consensus.cc:2258] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:14.210109 8372 tablet_service.cc:1467] Processing CreateTablet for tablet 9941dd00900c434baaba7b644880aeab (DEFAULT_TABLE table=test-table [id=10c753a558214e2194932b5b01d8d4f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:14.210119 8703 raft_consensus.cc:2229] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:14.210413 8372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9941dd00900c434baaba7b644880aeab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:14.210417 8703 raft_consensus.cc:2258] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:14.213240 8644 tablet_bootstrap.cc:492] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704: Bootstrap starting.
I20260809 04:40:14.214198 8644 tablet_bootstrap.cc:654] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:14.215341 8702 ts_tablet_manager.cc:1905] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:14.215837 8703 ts_tablet_manager.cc:1905] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:14.216315 8644 tablet_bootstrap.cc:492] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704: No bootstrap required, opened a new log
I20260809 04:40:14.216411 8644 ts_tablet_manager.cc:1397] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:14.216872 8644 raft_consensus.cc:348] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:14.216997 8644 raft_consensus.cc:374] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:14.217037 8644 raft_consensus.cc:729] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7931f4b9ad40c899fe1751ae813704, State: Initialized, Role: FOLLOWER
I20260809 04:40:14.217177 8644 consensus_queue.cc:260] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:14.217446 8644 ts_tablet_manager.cc:1428] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:14.221012 8702 ts_tablet_manager.cc:1918] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:14.224201 8703 ts_tablet_manager.cc:1918] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:14.224336 8703 log.cc:1196] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/23f3404bf34a409abacff43d31066ecb
I20260809 04:40:14.224756 8703 ts_tablet_manager.cc:1939] T 23f3404bf34a409abacff43d31066ecb P 1c7931f4b9ad40c899fe1751ae813704: Deleting consensus metadata
I20260809 04:40:14.226037 8031 catalog_manager.cc:4841] TS 1c7931f4b9ad40c899fe1751ae813704 (127.5.222.194:35219): tablet 23f3404bf34a409abacff43d31066ecb (table test-table [id=71eb59a6a5ae47a7a5572c6b5f5a772b]) successfully deleted
I20260809 04:40:14.226840 8702 log.cc:1196] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/23f3404bf34a409abacff43d31066ecb
I20260809 04:40:14.226922 8643 tablet_bootstrap.cc:492] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50: No bootstrap required, opened a new log
I20260809 04:40:14.227038 8643 ts_tablet_manager.cc:1397] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20260809 04:40:14.227274 8702 ts_tablet_manager.cc:1939] T 23f3404bf34a409abacff43d31066ecb P 4d4be658b5a54419b49eb76a8220b7a2: Deleting consensus metadata
I20260809 04:40:14.227722 8643 raft_consensus.cc:348] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:14.227862 8643 raft_consensus.cc:374] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:14.227905 8643 raft_consensus.cc:729] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666bc308874544f0bd5b536ca387ad50, State: Initialized, Role: FOLLOWER
I20260809 04:40:14.228080 8643 consensus_queue.cc:260] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:14.228487 8643 ts_tablet_manager.cc:1428] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 04:40:14.228722 8027 catalog_manager.cc:4841] TS 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095): tablet 23f3404bf34a409abacff43d31066ecb (table test-table [id=71eb59a6a5ae47a7a5572c6b5f5a772b]) successfully deleted
I20260809 04:40:14.229274 8642 tablet_bootstrap.cc:492] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2: No bootstrap required, opened a new log
I20260809 04:40:14.229372 8642 ts_tablet_manager.cc:1397] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.001s
I20260809 04:40:14.229920 8642 raft_consensus.cc:348] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:14.230058 8642 raft_consensus.cc:374] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:14.230108 8642 raft_consensus.cc:729] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4be658b5a54419b49eb76a8220b7a2, State: Initialized, Role: FOLLOWER
I20260809 04:40:14.230247 8642 consensus_queue.cc:260] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:14.230605 8642 ts_tablet_manager.cc:1428] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:14.234659 8678 raft_consensus.cc:482] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:14.234786 8678 raft_consensus.cc:504] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:14.235337 8678 leader_election.cc:288] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549), 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095)
I20260809 04:40:14.235745 8231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9941dd00900c434baaba7b644880aeab" candidate_uuid: "1c7931f4b9ad40c899fe1751ae813704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666bc308874544f0bd5b536ca387ad50" is_pre_election: true
I20260809 04:40:14.235909 8231 raft_consensus.cc:2454] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c7931f4b9ad40c899fe1751ae813704 in term 0.
I20260809 04:40:14.235841 8562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9941dd00900c434baaba7b644880aeab" candidate_uuid: "1c7931f4b9ad40c899fe1751ae813704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4be658b5a54419b49eb76a8220b7a2" is_pre_election: true
I20260809 04:40:14.236045 8562 raft_consensus.cc:2454] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c7931f4b9ad40c899fe1751ae813704 in term 0.
I20260809 04:40:14.236258 8321 leader_election.cc:302] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7931f4b9ad40c899fe1751ae813704, 666bc308874544f0bd5b536ca387ad50; no voters:
I20260809 04:40:14.236636 8678 raft_consensus.cc:2781] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:14.236722 8678 raft_consensus.cc:482] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:14.236784 8678 raft_consensus.cc:3037] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:14.238420 8678 raft_consensus.cc:504] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:14.238922 8678 leader_election.cc:288] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [CANDIDATE]: Term 1 election: Requested vote from peers 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549), 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095)
I20260809 04:40:14.239264 8231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9941dd00900c434baaba7b644880aeab" candidate_uuid: "1c7931f4b9ad40c899fe1751ae813704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666bc308874544f0bd5b536ca387ad50"
I20260809 04:40:14.239404 8231 raft_consensus.cc:3037] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:14.239463 8562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9941dd00900c434baaba7b644880aeab" candidate_uuid: "1c7931f4b9ad40c899fe1751ae813704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4be658b5a54419b49eb76a8220b7a2"
I20260809 04:40:14.239569 8562 raft_consensus.cc:3037] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:14.240623 8562 raft_consensus.cc:2454] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c7931f4b9ad40c899fe1751ae813704 in term 1.
I20260809 04:40:14.240656 8231 raft_consensus.cc:2454] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c7931f4b9ad40c899fe1751ae813704 in term 1.
I20260809 04:40:14.241006 8316 leader_election.cc:302] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7931f4b9ad40c899fe1751ae813704, 4d4be658b5a54419b49eb76a8220b7a2; no voters:
I20260809 04:40:14.241365 8678 raft_consensus.cc:2781] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:14.241495 8678 raft_consensus.cc:686] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 1 LEADER]: Becoming Leader. State: Replica: 1c7931f4b9ad40c899fe1751ae813704, State: Running, Role: LEADER
I20260809 04:40:14.241649 8678 consensus_queue.cc:237] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:14.243728 8047 catalog_manager.cc:5495] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c7931f4b9ad40c899fe1751ae813704 (127.5.222.194). New cstate: current_term: 1 leader_uuid: "1c7931f4b9ad40c899fe1751ae813704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20260809 04:40:14.263711 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.260504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52964 (local address 127.5.222.194:35219)
0809 04:40:14.260623 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:14.260627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:14.260741 (+ 114us) server_negotiation.cc:407] Connection header received
0809 04:40:14.260887 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:14.260889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:14.260929 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:14.261022 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:14.261312 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.261817 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.262662 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.262897 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.263012 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.263262 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:14.263337 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:14.263424 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:14.263513 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 04:40:14.263562 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 04:40:14.273072 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.270143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:35553 (local address 127.5.222.194:35219)
0809 04:40:14.270247 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:14.270251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:14.270273 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:40:14.270446 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:14.270448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:14.270485 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:14.270562 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:14.270767 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.271258 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.272038 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.272260 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.272363 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.272617 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:14.272686 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:14.272764 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:14.272843 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 04:40:14.272911 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":39}
I20260809 04:40:14.273072 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:14.270028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:35219 (local address 127.5.222.194:35553)
0809 04:40:14.270174 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:14.270195 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:14.270321 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:14.270579 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:14.270582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:14.270588 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:40:14.270666 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:14.270671 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.271360 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.271366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:14.271925 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:14.271932 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:14.272361 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:14.272364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:14.272483 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:14.272509 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:40:14.272766 (+ 257us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:40:14.272773 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 04:40:14.272837 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 04:40:14.272933 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":27}
I20260809 04:40:14.280473 8562 raft_consensus.cc:1261] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Refusing update from remote peer 1c7931f4b9ad40c899fe1751ae813704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:14.280902 8651 consensus_queue.cc:1035] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:14.281329 8231 raft_consensus.cc:1261] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Refusing update from remote peer 1c7931f4b9ad40c899fe1751ae813704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:14.281733 8678 consensus_queue.cc:1035] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 04:40:14.298544 8672 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 04:40:15.854436 8047 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:15.854583 8047 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:15.856989 8047 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4: Sending DeleteTablet for 3 replicas of tablet 9941dd00900c434baaba7b644880aeab
I20260809 04:40:15.857584 8210 tablet_service.cc:1514] Processing DeleteTablet for tablet 9941dd00900c434baaba7b644880aeab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:15 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:37334
I20260809 04:40:15.857796 8371 tablet_service.cc:1514] Processing DeleteTablet for tablet 9941dd00900c434baaba7b644880aeab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:15 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:52946
I20260809 04:40:15.857990 8736 tablet_replica.cc:330] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50: stopping tablet replica
I20260809 04:40:15.858127 8736 raft_consensus.cc:2229] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:15.858186 8737 tablet_replica.cc:330] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704: stopping tablet replica
I20260809 04:40:15.858270 8736 raft_consensus.cc:2258] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:15.858337 8737 raft_consensus.cc:2229] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:15.858660 8737 raft_consensus.cc:2258] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:15.858896 8736 ts_tablet_manager.cc:1905] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:15.859927 8542 tablet_service.cc:1514] Processing DeleteTablet for tablet 9941dd00900c434baaba7b644880aeab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:15 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:52624
I20260809 04:40:15.860319 8739 tablet_replica.cc:330] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2: stopping tablet replica
I20260809 04:40:15.860519 8739 raft_consensus.cc:2229] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:15.860688 8739 raft_consensus.cc:2258] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:15.861510 8736 ts_tablet_manager.cc:1918] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:40:15.861616 8736 log.cc:1196] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/wal/wals/9941dd00900c434baaba7b644880aeab
I20260809 04:40:15.861817 8737 ts_tablet_manager.cc:1905] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:15.861924 8736 ts_tablet_manager.cc:1939] T 9941dd00900c434baaba7b644880aeab P 666bc308874544f0bd5b536ca387ad50: Deleting consensus metadata
I20260809 04:40:15.862579 8739 ts_tablet_manager.cc:1905] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:15.862619 8031 catalog_manager.cc:4841] TS 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549): tablet 9941dd00900c434baaba7b644880aeab (table test-table [id=10c753a558214e2194932b5b01d8d4f5]) successfully deleted
I20260809 04:40:15.864503 8737 ts_tablet_manager.cc:1918] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:40:15.864567 8737 log.cc:1196] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/9941dd00900c434baaba7b644880aeab
I20260809 04:40:15.864655 8739 ts_tablet_manager.cc:1918] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:40:15.864740 8739 log.cc:1196] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/9941dd00900c434baaba7b644880aeab
I20260809 04:40:15.864915 8737 ts_tablet_manager.cc:1939] T 9941dd00900c434baaba7b644880aeab P 1c7931f4b9ad40c899fe1751ae813704: Deleting consensus metadata
I20260809 04:40:15.865026 8739 ts_tablet_manager.cc:1939] T 9941dd00900c434baaba7b644880aeab P 4d4be658b5a54419b49eb76a8220b7a2: Deleting consensus metadata
I20260809 04:40:15.865658 8031 catalog_manager.cc:4841] TS 1c7931f4b9ad40c899fe1751ae813704 (127.5.222.194:35219): tablet 9941dd00900c434baaba7b644880aeab (table test-table [id=10c753a558214e2194932b5b01d8d4f5]) successfully deleted
I20260809 04:40:15.865665 8027 catalog_manager.cc:4841] TS 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095): tablet 9941dd00900c434baaba7b644880aeab (table test-table [id=10c753a558214e2194932b5b01d8d4f5]) successfully deleted
Aug 09 04:40:15 dist-test-slave-vxj2 krb5kdc[7997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:40:15 dist-test-slave-vxj2 krb5kdc[7997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:15.895365 8747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:15.881001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40590 (local address 127.5.222.254:41677)
0809 04:40:15.881342 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:15.881353 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:15.881370 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:40:15.881533 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:15.881535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:15.881584 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:15.881684 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:15.882392 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:15.883001 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:15.883854 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:15.884099 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:15.884212 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:15.891938 (+ 7726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:15.891941 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:15.891944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:15.891974 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:15.893564 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:15.894013 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:15.894017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:15.894018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:15.894100 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:15.894382 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:15.894386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:15.894388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:15.894594 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:15.894740 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:15.895009 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 04:40:15.895158 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":106,"threads_started":1}
I20260809 04:40:15.898450 8047 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40590:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:15.898857 8047 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:15.903687 8210 tablet_service.cc:1467] Processing CreateTablet for tablet c0f242be6d924f5380f0d122a3a1667f (DEFAULT_TABLE table=test-table [id=c119dc4cbcef4041973dc6f0ebdc73c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:15.903779 8542 tablet_service.cc:1467] Processing CreateTablet for tablet c0f242be6d924f5380f0d122a3a1667f (DEFAULT_TABLE table=test-table [id=c119dc4cbcef4041973dc6f0ebdc73c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:15.903774 8371 tablet_service.cc:1467] Processing CreateTablet for tablet c0f242be6d924f5380f0d122a3a1667f (DEFAULT_TABLE table=test-table [id=c119dc4cbcef4041973dc6f0ebdc73c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:15.904006 8210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0f242be6d924f5380f0d122a3a1667f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:15.904039 8542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0f242be6d924f5380f0d122a3a1667f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:15.904098 8371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0f242be6d924f5380f0d122a3a1667f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:15.906788 8750 tablet_bootstrap.cc:492] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50: Bootstrap starting.
I20260809 04:40:15.907060 8748 tablet_bootstrap.cc:492] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2: Bootstrap starting.
I20260809 04:40:15.907248 8749 tablet_bootstrap.cc:492] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704: Bootstrap starting.
I20260809 04:40:15.907821 8750 tablet_bootstrap.cc:654] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:15.908058 8748 tablet_bootstrap.cc:654] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:15.908151 8749 tablet_bootstrap.cc:654] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:15.909217 8750 tablet_bootstrap.cc:492] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50: No bootstrap required, opened a new log
I20260809 04:40:15.909228 8748 tablet_bootstrap.cc:492] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2: No bootstrap required, opened a new log
I20260809 04:40:15.909313 8750 ts_tablet_manager.cc:1397] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:15.909313 8748 ts_tablet_manager.cc:1397] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:15.909763 8748 raft_consensus.cc:348] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:15.909894 8748 raft_consensus.cc:374] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:15.909948 8748 raft_consensus.cc:729] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4be658b5a54419b49eb76a8220b7a2, State: Initialized, Role: FOLLOWER
I20260809 04:40:15.909883 8750 raft_consensus.cc:348] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:15.909996 8750 raft_consensus.cc:374] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:15.910036 8750 raft_consensus.cc:729] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666bc308874544f0bd5b536ca387ad50, State: Initialized, Role: FOLLOWER
I20260809 04:40:15.910087 8748 consensus_queue.cc:260] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:15.910147 8750 consensus_queue.cc:260] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:15.910477 8748 ts_tablet_manager.cc:1428] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:15.910512 8750 ts_tablet_manager.cc:1428] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:15.911218 8749 tablet_bootstrap.cc:492] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704: No bootstrap required, opened a new log
I20260809 04:40:15.911314 8749 ts_tablet_manager.cc:1397] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 04:40:15.911775 8749 raft_consensus.cc:348] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:15.911931 8749 raft_consensus.cc:374] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:15.911979 8749 raft_consensus.cc:729] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7931f4b9ad40c899fe1751ae813704, State: Initialized, Role: FOLLOWER
I20260809 04:40:15.912109 8749 consensus_queue.cc:260] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:15.912386 8749 ts_tablet_manager.cc:1428] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 04:40:16.098012 8716 raft_consensus.cc:482] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:16.098224 8716 raft_consensus.cc:504] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:16.098774 8716 leader_election.cc:288] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549), 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095)
I20260809 04:40:16.099306 8562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0f242be6d924f5380f0d122a3a1667f" candidate_uuid: "1c7931f4b9ad40c899fe1751ae813704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4be658b5a54419b49eb76a8220b7a2" is_pre_election: true
I20260809 04:40:16.099342 8230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0f242be6d924f5380f0d122a3a1667f" candidate_uuid: "1c7931f4b9ad40c899fe1751ae813704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666bc308874544f0bd5b536ca387ad50" is_pre_election: true
I20260809 04:40:16.099486 8562 raft_consensus.cc:2454] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c7931f4b9ad40c899fe1751ae813704 in term 0.
I20260809 04:40:16.099485 8230 raft_consensus.cc:2454] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c7931f4b9ad40c899fe1751ae813704 in term 0.
I20260809 04:40:16.099886 8316 leader_election.cc:302] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7931f4b9ad40c899fe1751ae813704, 4d4be658b5a54419b49eb76a8220b7a2; no voters:
I20260809 04:40:16.100119 8716 raft_consensus.cc:2781] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:16.100189 8716 raft_consensus.cc:482] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:16.100226 8716 raft_consensus.cc:3037] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:16.101449 8716 raft_consensus.cc:504] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:16.101819 8716 leader_election.cc:288] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [CANDIDATE]: Term 1 election: Requested vote from peers 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549), 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095)
I20260809 04:40:16.102227 8562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0f242be6d924f5380f0d122a3a1667f" candidate_uuid: "1c7931f4b9ad40c899fe1751ae813704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4be658b5a54419b49eb76a8220b7a2"
I20260809 04:40:16.102241 8230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0f242be6d924f5380f0d122a3a1667f" candidate_uuid: "1c7931f4b9ad40c899fe1751ae813704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666bc308874544f0bd5b536ca387ad50"
I20260809 04:40:16.102356 8562 raft_consensus.cc:3037] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:16.102447 8230 raft_consensus.cc:3037] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:16.103459 8562 raft_consensus.cc:2454] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c7931f4b9ad40c899fe1751ae813704 in term 1.
I20260809 04:40:16.103463 8230 raft_consensus.cc:2454] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c7931f4b9ad40c899fe1751ae813704 in term 1.
I20260809 04:40:16.103806 8316 leader_election.cc:302] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7931f4b9ad40c899fe1751ae813704, 4d4be658b5a54419b49eb76a8220b7a2; no voters:
I20260809 04:40:16.104068 8716 raft_consensus.cc:2781] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:16.104182 8716 raft_consensus.cc:686] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 1 LEADER]: Becoming Leader. State: Replica: 1c7931f4b9ad40c899fe1751ae813704, State: Running, Role: LEADER
I20260809 04:40:16.104333 8716 consensus_queue.cc:237] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:16.106424 8043 catalog_manager.cc:5495] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c7931f4b9ad40c899fe1751ae813704 (127.5.222.194). New cstate: current_term: 1 leader_uuid: "1c7931f4b9ad40c899fe1751ae813704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20260809 04:40:16.151541 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:16.148079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.5.222.194:35219)
0809 04:40:16.148433 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:16.148441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:16.148452 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:40:16.148486 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:16.148488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:16.148526 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:16.148618 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:16.148935 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.149597 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.150448 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.150681 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.150793 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:16.151063 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:16.151132 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:16.151214 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:16.151334 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 04:40:16.151386 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":152,"threads_started":1}
I20260809 04:40:16.157416 8562 raft_consensus.cc:1261] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Refusing update from remote peer 1c7931f4b9ad40c899fe1751ae813704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:16.157433 8230 raft_consensus.cc:1261] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Refusing update from remote peer 1c7931f4b9ad40c899fe1751ae813704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:16.157837 8716 consensus_queue.cc:1035] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:16.157990 8722 consensus_queue.cc:1035] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:16.193334 8048 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:16.193493 8048 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:16.195577 8048 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4: Sending DeleteTablet for 3 replicas of tablet c0f242be6d924f5380f0d122a3a1667f
I20260809 04:40:16.196214 8210 tablet_service.cc:1514] Processing DeleteTablet for tablet c0f242be6d924f5380f0d122a3a1667f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:16 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:37334
I20260809 04:40:16.196398 8736 tablet_replica.cc:330] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50: stopping tablet replica
I20260809 04:40:16.196362 8542 tablet_service.cc:1514] Processing DeleteTablet for tablet c0f242be6d924f5380f0d122a3a1667f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:16 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:52624
I20260809 04:40:16.196390 8371 tablet_service.cc:1514] Processing DeleteTablet for tablet c0f242be6d924f5380f0d122a3a1667f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:16 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:52946
I20260809 04:40:16.196509 8736 raft_consensus.cc:2229] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:16.196566 8737 tablet_replica.cc:330] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704: stopping tablet replica
I20260809 04:40:16.196597 8739 tablet_replica.cc:330] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2: stopping tablet replica
I20260809 04:40:16.196626 8736 raft_consensus.cc:2258] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:16.196671 8737 raft_consensus.cc:2229] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:16.196724 8739 raft_consensus.cc:2229] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:16.196830 8739 raft_consensus.cc:2258] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:16.196821 8737 raft_consensus.cc:2258] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:16.197505 8737 ts_tablet_manager.cc:1905] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:16.197592 8736 ts_tablet_manager.cc:1905] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:16.199635 8736 ts_tablet_manager.cc:1918] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:16.199644 8737 ts_tablet_manager.cc:1918] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:16.199733 8736 log.cc:1196] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-0/wal/wals/c0f242be6d924f5380f0d122a3a1667f
I20260809 04:40:16.199725 8737 log.cc:1196] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/c0f242be6d924f5380f0d122a3a1667f
I20260809 04:40:16.199988 8736 ts_tablet_manager.cc:1939] T c0f242be6d924f5380f0d122a3a1667f P 666bc308874544f0bd5b536ca387ad50: Deleting consensus metadata
I20260809 04:40:16.199990 8737 ts_tablet_manager.cc:1939] T c0f242be6d924f5380f0d122a3a1667f P 1c7931f4b9ad40c899fe1751ae813704: Deleting consensus metadata
I20260809 04:40:16.200675 8031 catalog_manager.cc:4841] TS 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549): tablet c0f242be6d924f5380f0d122a3a1667f (table test-table [id=c119dc4cbcef4041973dc6f0ebdc73c8]) successfully deleted
I20260809 04:40:16.200850 8031 catalog_manager.cc:4841] TS 1c7931f4b9ad40c899fe1751ae813704 (127.5.222.194:35219): tablet c0f242be6d924f5380f0d122a3a1667f (table test-table [id=c119dc4cbcef4041973dc6f0ebdc73c8]) successfully deleted
I20260809 04:40:16.201095 8739 ts_tablet_manager.cc:1905] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:16.203048 8739 ts_tablet_manager.cc:1918] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:16.203141 8739 log.cc:1196] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/c0f242be6d924f5380f0d122a3a1667f
I20260809 04:40:16.203440 8739 ts_tablet_manager.cc:1939] T c0f242be6d924f5380f0d122a3a1667f P 4d4be658b5a54419b49eb76a8220b7a2: Deleting consensus metadata
I20260809 04:40:16.203727 8048 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40590:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:16.204113 8048 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:16.204324 8027 catalog_manager.cc:4841] TS 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095): tablet c0f242be6d924f5380f0d122a3a1667f (table test-table [id=c119dc4cbcef4041973dc6f0ebdc73c8]) successfully deleted
I20260809 04:40:16.208735 8210 tablet_service.cc:1467] Processing CreateTablet for tablet b1bfebe7a29847c9b205bde93d4e55e6 (DEFAULT_TABLE table=test-table [id=6d10692aed334e40bade9fc2464885b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:16.208770 8371 tablet_service.cc:1467] Processing CreateTablet for tablet b1bfebe7a29847c9b205bde93d4e55e6 (DEFAULT_TABLE table=test-table [id=6d10692aed334e40bade9fc2464885b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:16.208877 8542 tablet_service.cc:1467] Processing CreateTablet for tablet b1bfebe7a29847c9b205bde93d4e55e6 (DEFAULT_TABLE table=test-table [id=6d10692aed334e40bade9fc2464885b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:16.209033 8210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1bfebe7a29847c9b205bde93d4e55e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:16.209051 8371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1bfebe7a29847c9b205bde93d4e55e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:16.209108 8542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1bfebe7a29847c9b205bde93d4e55e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:16.211068 8749 tablet_bootstrap.cc:492] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704: Bootstrap starting.
I20260809 04:40:16.211441 8750 tablet_bootstrap.cc:492] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50: Bootstrap starting.
I20260809 04:40:16.211483 8748 tablet_bootstrap.cc:492] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2: Bootstrap starting.
I20260809 04:40:16.211927 8749 tablet_bootstrap.cc:654] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:16.212241 8748 tablet_bootstrap.cc:654] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:16.212466 8750 tablet_bootstrap.cc:654] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:16.213331 8749 tablet_bootstrap.cc:492] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704: No bootstrap required, opened a new log
I20260809 04:40:16.213439 8749 ts_tablet_manager.cc:1397] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:16.213840 8750 tablet_bootstrap.cc:492] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50: No bootstrap required, opened a new log
I20260809 04:40:16.213941 8750 ts_tablet_manager.cc:1397] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:16.213898 8749 raft_consensus.cc:348] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:16.214021 8749 raft_consensus.cc:374] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:16.214071 8749 raft_consensus.cc:729] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7931f4b9ad40c899fe1751ae813704, State: Initialized, Role: FOLLOWER
I20260809 04:40:16.214190 8748 tablet_bootstrap.cc:492] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2: No bootstrap required, opened a new log
I20260809 04:40:16.214207 8749 consensus_queue.cc:260] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:16.214296 8748 ts_tablet_manager.cc:1397] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 04:40:16.214491 8749 ts_tablet_manager.cc:1428] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:16.214437 8750 raft_consensus.cc:348] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:16.214545 8750 raft_consensus.cc:374] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:16.214589 8750 raft_consensus.cc:729] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666bc308874544f0bd5b536ca387ad50, State: Initialized, Role: FOLLOWER
I20260809 04:40:16.214689 8748 raft_consensus.cc:348] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:16.214715 8750 consensus_queue.cc:260] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:16.214826 8748 raft_consensus.cc:374] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:16.214869 8748 raft_consensus.cc:729] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4be658b5a54419b49eb76a8220b7a2, State: Initialized, Role: FOLLOWER
I20260809 04:40:16.214982 8748 consensus_queue.cc:260] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:16.215260 8748 ts_tablet_manager.cc:1428] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:16.216650 8750 ts_tablet_manager.cc:1428] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:16.405445 8756 raft_consensus.cc:482] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:16.405591 8756 raft_consensus.cc:504] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:16.406903 8756 leader_election.cc:288] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c7931f4b9ad40c899fe1751ae813704 (127.5.222.194:35219), 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095)
I20260809 04:40:16.411700 8771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:16.407771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:56481 (local address 127.5.222.195:38095)
0809 04:40:16.408030 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:16.408042 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:16.408053 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:40:16.408093 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:16.408095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:16.408155 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:16.408248 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:16.408509 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.409271 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.410370 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.410942 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.411063 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:16.411115 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:16.411470 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 04:40:16.411567 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":79,"threads_started":1}
I20260809 04:40:16.411692 8769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:16.407141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:38095 (local address 127.5.222.193:56481)
0809 04:40:16.407760 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:16.407776 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:16.407842 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:16.408273 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:16.408276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:16.408293 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:16.408389 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:16.408394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.409412 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.409421 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:16.410238 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:16.410247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.411076 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.411081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:16.411233 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:16.411260 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:40:16.411335 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 04:40:16.411402 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":117,"threads_started":1}
I20260809 04:40:16.411729 8770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:16.407115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:35219 (local address 127.5.222.193:58929)
0809 04:40:16.407442 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:16.407458 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:16.407557 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:16.407933 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:16.407936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:16.407947 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:16.408081 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:16.408086 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.409121 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.409125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:16.410493 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:16.410504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.411330 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.411334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:16.411448 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:16.411470 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:40:16.411541 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 04:40:16.411602 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":93,"threads_started":1}
I20260809 04:40:16.412187 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:16.407488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:58929 (local address 127.5.222.194:35219)
0809 04:40:16.407609 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:16.407614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:16.407628 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:40:16.407685 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:16.407691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:16.407746 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:16.407838 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:16.408198 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.409002 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.410619 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.411208 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.411338 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:16.411387 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:16.411988 (+ 601us) server_negotiation.cc:299] Negotiation successful
0809 04:40:16.412074 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:40:16.412487 8562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1bfebe7a29847c9b205bde93d4e55e6" candidate_uuid: "666bc308874544f0bd5b536ca387ad50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4be658b5a54419b49eb76a8220b7a2" is_pre_election: true
I20260809 04:40:16.412565 8394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1bfebe7a29847c9b205bde93d4e55e6" candidate_uuid: "666bc308874544f0bd5b536ca387ad50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7931f4b9ad40c899fe1751ae813704" is_pre_election: true
I20260809 04:40:16.412642 8562 raft_consensus.cc:2454] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666bc308874544f0bd5b536ca387ad50 in term 0.
I20260809 04:40:16.412693 8394 raft_consensus.cc:2454] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666bc308874544f0bd5b536ca387ad50 in term 0.
I20260809 04:40:16.413626 8159 leader_election.cc:302] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c7931f4b9ad40c899fe1751ae813704, 666bc308874544f0bd5b536ca387ad50; no voters:
I20260809 04:40:16.413914 8756 raft_consensus.cc:2781] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:16.413982 8756 raft_consensus.cc:482] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:16.414017 8756 raft_consensus.cc:3037] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:16.415563 8756 raft_consensus.cc:504] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:16.415927 8756 leader_election.cc:288] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [CANDIDATE]: Term 1 election: Requested vote from peers 1c7931f4b9ad40c899fe1751ae813704 (127.5.222.194:35219), 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095)
I20260809 04:40:16.416297 8394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1bfebe7a29847c9b205bde93d4e55e6" candidate_uuid: "666bc308874544f0bd5b536ca387ad50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c7931f4b9ad40c899fe1751ae813704"
I20260809 04:40:16.416312 8562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1bfebe7a29847c9b205bde93d4e55e6" candidate_uuid: "666bc308874544f0bd5b536ca387ad50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4be658b5a54419b49eb76a8220b7a2"
I20260809 04:40:16.416428 8394 raft_consensus.cc:3037] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:16.416447 8562 raft_consensus.cc:3037] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:16.417668 8562 raft_consensus.cc:2454] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666bc308874544f0bd5b536ca387ad50 in term 1.
I20260809 04:40:16.417668 8394 raft_consensus.cc:2454] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666bc308874544f0bd5b536ca387ad50 in term 1.
I20260809 04:40:16.418028 8156 leader_election.cc:302] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c7931f4b9ad40c899fe1751ae813704, 4d4be658b5a54419b49eb76a8220b7a2, 666bc308874544f0bd5b536ca387ad50; no voters:
I20260809 04:40:16.418191 8756 raft_consensus.cc:2781] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:16.418485 8756 raft_consensus.cc:686] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [term 1 LEADER]: Becoming Leader. State: Replica: 666bc308874544f0bd5b536ca387ad50, State: Running, Role: LEADER
I20260809 04:40:16.418748 8756 consensus_queue.cc:237] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } }
I20260809 04:40:16.421331 8048 catalog_manager.cc:5495] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666bc308874544f0bd5b536ca387ad50 (127.5.222.193). New cstate: current_term: 1 leader_uuid: "666bc308874544f0bd5b536ca387ad50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666bc308874544f0bd5b536ca387ad50" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 35549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20260809 04:40:16.459664 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:16.455841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.5.222.193:35549)
0809 04:40:16.456112 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:16.456119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:16.456130 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:40:16.456201 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:16.456203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:16.456256 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:16.456375 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:16.456578 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.457291 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.458134 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.458406 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.458506 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:16.458730 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:16.459218 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:16.459320 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:16.459378 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 04:40:16.459429 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":119,"threads_started":1}
I20260809 04:40:16.469477 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:16.466278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:53687 (local address 127.5.222.194:35219)
0809 04:40:16.466424 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:16.466427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:16.466610 (+ 183us) server_negotiation.cc:407] Connection header received
0809 04:40:16.466661 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:16.466664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:16.466709 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:16.466820 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:16.467154 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.467652 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.468472 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.468684 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.468812 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:16.469086 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:16.469169 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:16.469226 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:16.469324 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 04:40:16.469372 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:40:16.469521 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:16.466182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:35219 (local address 127.5.222.193:53687)
0809 04:40:16.466503 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:16.466517 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:16.466602 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:16.466837 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:16.466839 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:16.466844 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:40:16.467019 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:16.467026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.467753 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.467756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:16.468362 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:16.468370 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.468845 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.468848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:16.468922 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:16.468963 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:40:16.469247 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:40:16.469252 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:40:16.469301 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 04:40:16.469368 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":107,"threads_started":1}
I20260809 04:40:16.475514 8562 raft_consensus.cc:1261] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Refusing update from remote peer 666bc308874544f0bd5b536ca387ad50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:16.475514 8394 raft_consensus.cc:1261] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Refusing update from remote peer 666bc308874544f0bd5b536ca387ad50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:16.476070 8774 consensus_queue.cc:1035] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c7931f4b9ad40c899fe1751ae813704" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 35219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:16.476292 8756 consensus_queue.cc:1035] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4be658b5a54419b49eb76a8220b7a2" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:16.494061 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:16.490443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:38375 (local address 127.5.222.193:35549)
0809 04:40:16.490583 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:16.490586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:16.490596 (+ 10us) server_negotiation.cc:407] Connection header received
0809 04:40:16.490633 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:16.490634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:16.490701 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:16.490781 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:16.491030 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.491701 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.492650 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.492960 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.493079 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:16.493500 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:16.493586 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:16.493681 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:16.493833 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 04:40:16.493909 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:40:16.494073 8784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:16.490006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:35549 (local address 127.5.222.194:38375)
0809 04:40:16.490300 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:16.490314 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:16.490411 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:16.490807 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:16.490809 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:16.490818 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:40:16.490913 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:16.490918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.491819 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.491824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:16.492494 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:16.492503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:16.493114 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:16.493118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:16.493278 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:16.493355 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:40:16.493687 (+ 332us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:40:16.493698 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 04:40:16.493814 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 04:40:16.493929 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":125,"threads_started":1}
W20260809 04:40:16.494861 8718 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 04:40:18.054647 8048 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:18.054809 8048 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:18.057256 8048 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e163dcdbf7454fa29f967756cf6590c4: Sending DeleteTablet for 3 replicas of tablet b1bfebe7a29847c9b205bde93d4e55e6
I20260809 04:40:18.057775 8210 tablet_service.cc:1514] Processing DeleteTablet for tablet b1bfebe7a29847c9b205bde93d4e55e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:18 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:37334
I20260809 04:40:18.057847 8371 tablet_service.cc:1514] Processing DeleteTablet for tablet b1bfebe7a29847c9b205bde93d4e55e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:18 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:52946
I20260809 04:40:18.057904 8542 tablet_service.cc:1514] Processing DeleteTablet for tablet b1bfebe7a29847c9b205bde93d4e55e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:18 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:52624
I20260809 04:40:18.058216 8810 tablet_replica.cc:330] T b1bfebe7a29847c9b205bde93d4e55e6 P 666bc308874544f0bd5b536ca387ad50: stopping tablet replica
I20260809 04:40:18.058288 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 8137
I20260809 04:40:18.060787 8813 tablet_replica.cc:330] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2: stopping tablet replica
I20260809 04:40:18.060966 8813 raft_consensus.cc:2229] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:18.061090 8813 raft_consensus.cc:2258] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:18.058205 8811 tablet_replica.cc:330] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704: stopping tablet replica
I20260809 04:40:18.062186 8811 raft_consensus.cc:2229] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:18.062283 8811 raft_consensus.cc:2258] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:18.062832 8813 ts_tablet_manager.cc:1905] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:18.064989 8813 ts_tablet_manager.cc:1918] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:40:18.065078 8813 log.cc:1196] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/b1bfebe7a29847c9b205bde93d4e55e6
I20260809 04:40:18.065310 8811 ts_tablet_manager.cc:1905] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:18.065442 8813 ts_tablet_manager.cc:1939] T b1bfebe7a29847c9b205bde93d4e55e6 P 4d4be658b5a54419b49eb76a8220b7a2: Deleting consensus metadata
I20260809 04:40:18.066295 8027 catalog_manager.cc:4841] TS 4d4be658b5a54419b49eb76a8220b7a2 (127.5.222.195:38095): tablet b1bfebe7a29847c9b205bde93d4e55e6 (table test-table [id=6d10692aed334e40bade9fc2464885b4]) successfully deleted
I20260809 04:40:18.067596 8811 ts_tablet_manager.cc:1918] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:40:18.067693 8811 log.cc:1196] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/b1bfebe7a29847c9b205bde93d4e55e6
I20260809 04:40:18.067924 8811 ts_tablet_manager.cc:1939] T b1bfebe7a29847c9b205bde93d4e55e6 P 1c7931f4b9ad40c899fe1751ae813704: Deleting consensus metadata
I20260809 04:40:18.068678 8031 catalog_manager.cc:4841] TS 1c7931f4b9ad40c899fe1751ae813704 (127.5.222.194:35219): tablet b1bfebe7a29847c9b205bde93d4e55e6 (table test-table [id=6d10692aed334e40bade9fc2464885b4]) successfully deleted
W20260809 04:40:18.072476 8031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:35549 (error 108)
I20260809 04:40:18.072713 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 8298
I20260809 04:40:18.075498 8815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:18.075068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:35549 (local address 127.0.0.1:56732)
0809 04:40:18.075334 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:18.075404 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:35549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":83,"threads_started":1}
W20260809 04:40:18.075695 8031 catalog_manager.cc:4584] TS 666bc308874544f0bd5b536ca387ad50 (127.5.222.193:35549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1bfebe7a29847c9b205bde93d4e55e6: Network error: Client connection negotiation failed: client connection to 127.5.222.193:35549: connect: Connection refused (error 111)
I20260809 04:40:18.080582 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 8461
I20260809 04:40:18.089210 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 8011
2026-08-09T04:40:18Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6898 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:18 dist-test-slave-vxj2 krb5kdc[8818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:18 dist-test-slave-vxj2 krb5kdc[8818](info): set up 2 sockets
Aug 09 04:40:18 dist-test-slave-vxj2 krb5kdc[8818](info): commencing operation
krb5kdc: starting...
W20260809 04:40:19.272011 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:19 dist-test-slave-vxj2 krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:40:19.408181 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:36427
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:36427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:19.518605 8837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:19.518918 8837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:19.518998 8837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:19.523003 8837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:19.523094 8837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:19.523128 8837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:19.523167 8837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:19.523202 8837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:19.528627 8837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:36427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:36427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:19.530028 8837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:19.531073 8837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:19.539714 8846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:19.539716 8844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:19.539989 8848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:19.539980 8837 server_base.cc:1034] running on GCE node
I20260809 04:40:19.540637 8837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:19.541754 8837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:19.542918 8837 hybrid_clock.cc:648] HybridClock initialized: now 1786250419542892 us; error 43 us; skew 500 ppm
Aug 09 04:40:19 dist-test-slave-vxj2 krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250419, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:19.545754 8837 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:40:19.546592 8837 webserver.cc:460] Webserver started at http://127.5.222.254:42083/ using document root <none> and password file <none>
I20260809 04:40:19.547091 8837 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:19.547147 8837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:19.547315 8837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:19.549306 8837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "731c5b9fca6544088463b04f0835cc63"
format_stamp: "Formatted at 2026-08-09 04:40:19 on dist-test-slave-vxj2"
server_key: "aef7a3f8310fadf8b887b0884f77d733"
server_key_iv: "c611f584c2071b74ce2496ede178aabd"
server_key_version: "encryptionkey@0"
I20260809 04:40:19.549830 8837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "731c5b9fca6544088463b04f0835cc63"
format_stamp: "Formatted at 2026-08-09 04:40:19 on dist-test-slave-vxj2"
server_key: "aef7a3f8310fadf8b887b0884f77d733"
server_key_iv: "c611f584c2071b74ce2496ede178aabd"
server_key_version: "encryptionkey@0"
I20260809 04:40:19.553354 8837 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:40:19.555696 8856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:19.556949 8837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:19.557117 8837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "731c5b9fca6544088463b04f0835cc63"
format_stamp: "Formatted at 2026-08-09 04:40:19 on dist-test-slave-vxj2"
server_key: "aef7a3f8310fadf8b887b0884f77d733"
server_key_iv: "c611f584c2071b74ce2496ede178aabd"
server_key_version: "encryptionkey@0"
I20260809 04:40:19.557304 8837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:19.569428 8837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:19.572050 8837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:19.572252 8837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:19.579602 8837 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:36427
I20260809 04:40:19.579630 8912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:36427 every 8 connection(s)
I20260809 04:40:19.580574 8837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:19.583298 8914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:19.584394 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 8837
I20260809 04:40:19.584522 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:19.584857 6011 external_mini_cluster.cc:1428] Setting key 84dd89d21b2587d292ad9aa2655dfd19
I20260809 04:40:19.590061 8914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63: Bootstrap starting.
I20260809 04:40:19.592622 8914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:19.593510 8914 log.cc:824] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:40:19 dist-test-slave-vxj2 krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:19.595842 8914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63: No bootstrap required, opened a new log
I20260809 04:40:19.599874 8914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731c5b9fca6544088463b04f0835cc63" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36427 } }
I20260809 04:40:19.600121 8914 raft_consensus.cc:374] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:19.600209 8914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 731c5b9fca6544088463b04f0835cc63, State: Initialized, Role: FOLLOWER
I20260809 04:40:19.600660 8914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731c5b9fca6544088463b04f0835cc63" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36427 } }
I20260809 04:40:19.600814 8914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:19.600838 8919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:19.586334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.5.222.254:36427)
0809 04:40:19.586757 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:19.586777 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:19.586820 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:40:19.587655 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:19.587679 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:19.588095 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:19.588467 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:19.589290 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:19.590710 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:19.591615 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:19.592276 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:19.592459 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:19.595572 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:19.595578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:19.595590 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:19.595618 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:19.597703 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:19.598253 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:19.598258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:19.598262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:19.598383 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:19.598678 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:19.598680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:19.598681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:19.599082 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:19.599267 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:19.599542 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 04:40:19.599857 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":119,"threads_started":1}
I20260809 04:40:19.600903 8914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:19.601011 8914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:19.602023 8914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731c5b9fca6544088463b04f0835cc63" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36427 } }
I20260809 04:40:19.602380 8914 leader_election.cc:302] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 731c5b9fca6544088463b04f0835cc63; no voters:
I20260809 04:40:19.602741 8914 leader_election.cc:288] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:19.602866 8921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:19.603103 8921 raft_consensus.cc:686] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [term 1 LEADER]: Becoming Leader. State: Replica: 731c5b9fca6544088463b04f0835cc63, State: Running, Role: LEADER
I20260809 04:40:19.603430 8921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731c5b9fca6544088463b04f0835cc63" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36427 } }
I20260809 04:40:19.603984 8914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:19.605022 8923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 731c5b9fca6544088463b04f0835cc63. Latest consensus state: current_term: 1 leader_uuid: "731c5b9fca6544088463b04f0835cc63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731c5b9fca6544088463b04f0835cc63" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36427 } } }
I20260809 04:40:19.605177 8923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:19.605505 8922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "731c5b9fca6544088463b04f0835cc63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731c5b9fca6544088463b04f0835cc63" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36427 } } }
I20260809 04:40:19.605576 8932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:19.605692 8922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:19.608603 8932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:19.613888 8932 catalog_manager.cc:1347] Generated new cluster ID: e202e37a47894584a527647ea6cf4364
I20260809 04:40:19.613958 8932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:19.627193 8932 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:19.628433 8932 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:19.634634 8932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 731c5b9fca6544088463b04f0835cc63: Generated new TSK 0
I20260809 04:40:19.635371 8932 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:40:19.664081 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:36427
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:19.778468 8945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:19.778709 8945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:19.778782 8945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:19.782902 8945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:19.782961 8945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:19.783020 8945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:19.788196 8945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:36427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:19.789391 8945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:19.790439 8945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:19.800181 8950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:19.800215 8951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:19.800172 8954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:19.800509 8945 server_base.cc:1034] running on GCE node
I20260809 04:40:19.801075 8945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:19.801802 8945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:19.802975 8945 hybrid_clock.cc:648] HybridClock initialized: now 1786250419802954 us; error 31 us; skew 500 ppm
Aug 09 04:40:19 dist-test-slave-vxj2 krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250419, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:19.806020 8945 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:40:19.806849 8945 webserver.cc:460] Webserver started at http://127.5.222.193:33453/ using document root <none> and password file <none>
I20260809 04:40:19.807376 8945 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:19.807443 8945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:19.807619 8945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:19.809679 8945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "e5680e47703c4fd49325f68e73cc3f34"
format_stamp: "Formatted at 2026-08-09 04:40:19 on dist-test-slave-vxj2"
server_key: "6c283621dd16851f75b7b45993656504"
server_key_iv: "fe42b00d6b4736d86ce294db5215e6ef"
server_key_version: "encryptionkey@0"
I20260809 04:40:19.810240 8945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "e5680e47703c4fd49325f68e73cc3f34"
format_stamp: "Formatted at 2026-08-09 04:40:19 on dist-test-slave-vxj2"
server_key: "6c283621dd16851f75b7b45993656504"
server_key_iv: "fe42b00d6b4736d86ce294db5215e6ef"
server_key_version: "encryptionkey@0"
I20260809 04:40:19.813941 8945 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:40:19.816291 8962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:19.817552 8945 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:40:19.817670 8945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "e5680e47703c4fd49325f68e73cc3f34"
format_stamp: "Formatted at 2026-08-09 04:40:19 on dist-test-slave-vxj2"
server_key: "6c283621dd16851f75b7b45993656504"
server_key_iv: "fe42b00d6b4736d86ce294db5215e6ef"
server_key_version: "encryptionkey@0"
I20260809 04:40:19.817771 8945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:19.831180 8945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:19.834159 8945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:19.834359 8945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:19.834954 8945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:19.835763 8945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:19.835809 8945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:19.835875 8945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:19.835933 8945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:19.844370 8945 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:40469
I20260809 04:40:19.844436 9090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:40469 every 8 connection(s)
I20260809 04:40:19.845320 8945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:19.850670 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 8945
I20260809 04:40:19.850790 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:19.851083 6011 external_mini_cluster.cc:1428] Setting key 46021c0bf73caf355f9d9e73b94f4f2e
Aug 09 04:40:19 dist-test-slave-vxj2 krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250419, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:19.859748 8919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:19.847131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:59915 (local address 127.5.222.254:36427)
0809 04:40:19.847289 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:19.847293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:19.848036 (+ 743us) server_negotiation.cc:407] Connection header received
0809 04:40:19.849488 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:19.849491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:19.849550 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:19.849658 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:19.850969 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:19.851582 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:19.852934 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:19.853220 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:19.853305 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:19.856058 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:19.856061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:19.856063 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:19.856087 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:19.857562 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:19.858228 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:19.858231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:19.858232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:19.858300 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:19.858696 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:19.858698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:19.858699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:19.858859 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:19.858932 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:19.859536 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 04:40:19.859633 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:40:19.860924 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:19.847481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:36427 (local address 127.5.222.193:59915)
0809 04:40:19.847895 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:19.847936 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:19.849285 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:19.849832 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:19.849839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:19.850169 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:19.850800 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:19.850823 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:19.851751 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:19.851754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:19.852806 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:19.852820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:19.853372 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:19.853379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:19.853614 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:19.854278 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:19.854304 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:19.855899 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:19.857693 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:19.857712 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:19.857729 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:19.858117 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:19.858395 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:19.858400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:19.858405 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:19.858598 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:19.858966 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:19.858974 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:19.859282 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 04:40:19.859505 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 04:40:19.859852 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
I20260809 04:40:19.862183 9091 heartbeater.cc:347] Connected to a master server at 127.5.222.254:36427
I20260809 04:40:19.862386 9091 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:19.862911 9091 heartbeater.cc:511] Master 127.5.222.254:36427 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:40:19.864498 8874 ts_manager.cc:194] Registered new tserver with Master: e5680e47703c4fd49325f68e73cc3f34 (127.5.222.193:40469)
I20260809 04:40:19.866158 8874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:59915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:40:19.880460 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:36427
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:19.991606 9096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:19.991936 9096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:19.992022 9096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:19.996688 9096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:19.996789 9096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:19.996904 9096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:20.002291 9096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:36427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:20.003424 9096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:20.004585 9096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:20.014950 9103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:20.014940 9105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:20.014959 9102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:20.015193 9096 server_base.cc:1034] running on GCE node
I20260809 04:40:20.015714 9096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:20.016388 9096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:20.017581 9096 hybrid_clock.cc:648] HybridClock initialized: now 1786250420017572 us; error 31 us; skew 500 ppm
Aug 09 04:40:20 dist-test-slave-vxj2 krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250420, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:20.020412 9096 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:40:20.021453 9096 webserver.cc:460] Webserver started at http://127.5.222.194:34723/ using document root <none> and password file <none>
I20260809 04:40:20.022094 9096 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:20.022197 9096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:20.022409 9096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:20.024492 9096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "819a9cc4399a4a479cf0253fc3b66979"
format_stamp: "Formatted at 2026-08-09 04:40:20 on dist-test-slave-vxj2"
server_key: "15685dfca5d7c36371872fb95de45769"
server_key_iv: "e6ae99749dd5a46ea93c23ab9a8b5889"
server_key_version: "encryptionkey@0"
I20260809 04:40:20.025084 9096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "819a9cc4399a4a479cf0253fc3b66979"
format_stamp: "Formatted at 2026-08-09 04:40:20 on dist-test-slave-vxj2"
server_key: "15685dfca5d7c36371872fb95de45769"
server_key_iv: "e6ae99749dd5a46ea93c23ab9a8b5889"
server_key_version: "encryptionkey@0"
I20260809 04:40:20.028714 9096 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:40:20.031144 9115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:20.032371 9096 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:40:20.032531 9096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "819a9cc4399a4a479cf0253fc3b66979"
format_stamp: "Formatted at 2026-08-09 04:40:20 on dist-test-slave-vxj2"
server_key: "15685dfca5d7c36371872fb95de45769"
server_key_iv: "e6ae99749dd5a46ea93c23ab9a8b5889"
server_key_version: "encryptionkey@0"
I20260809 04:40:20.032649 9096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:20.048660 9096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:20.052260 9096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:20.052493 9096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:20.053143 9096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:20.054096 9096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:20.054208 9096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:20.054311 9096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:20.054368 9096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:20.064460 9096 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:34709
I20260809 04:40:20.064491 9243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:34709 every 8 connection(s)
I20260809 04:40:20.065500 9096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:20.066699 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 9096
I20260809 04:40:20.066835 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:20.067143 6011 external_mini_cluster.cc:1428] Setting key 3f4277d68ffde9495bad059377ce7d43
Aug 09 04:40:20 dist-test-slave-vxj2 krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250420, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
I20260809 04:40:20.080163 8919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:20.067433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:42663 (local address 127.5.222.254:36427)
0809 04:40:20.067611 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:20.067616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:20.068545 (+ 929us) server_negotiation.cc:407] Connection header received
0809 04:40:20.070013 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:20.070016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:20.070072 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:20.070181 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:20.071341 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:20.072169 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:20.073572 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:20.073809 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:20.073926 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:20.076399 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:20.076403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:20.076406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:20.076437 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:20.078001 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:20.078536 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:20.078542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:20.078543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:20.078625 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:20.079006 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:20.079011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:20.079013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:20.079217 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:20.079330 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:20.079857 (+ 527us) server_negotiation.cc:299] Negotiation successful
0809 04:40:20.079984 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:40:20.080957 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:20.067923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:36427 (local address 127.5.222.194:42663)
0809 04:40:20.068404 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:20.068458 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:20.069788 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:20.070351 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:20.070358 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:20.070708 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:20.071194 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:20.071207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:20.072310 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:20.072313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:20.073440 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:20.073452 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:20.073966 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:20.073970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:20.074158 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:20.074769 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:20.074786 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:20.076228 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:20.078134 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:20.078141 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:20.078155 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:20.078428 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:20.078762 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:20.078764 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:20.078766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:20.078906 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:20.079327 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:20.079333 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:20.079578 (+ 245us) client_negotiation.cc:769] Sending connection context
0809 04:40:20.079777 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 04:40:20.080103 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":142,"threads_started":1}
I20260809 04:40:20.082261 9244 heartbeater.cc:347] Connected to a master server at 127.5.222.254:36427
I20260809 04:40:20.082540 9244 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:20.083144 9244 heartbeater.cc:511] Master 127.5.222.254:36427 requested a full tablet report, sending...
I20260809 04:40:20.084324 8874 ts_manager.cc:194] Registered new tserver with Master: 819a9cc4399a4a479cf0253fc3b66979 (127.5.222.194:34709)
I20260809 04:40:20.084925 8874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:42663
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:40:20.096725 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:36427
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:20.207931 9249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:20.208179 9249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:20.208222 9249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:20.212430 9249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:20.212491 9249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:20.212549 9249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:20.217803 9249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:36427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:20.218789 9249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:20.219746 9249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:20.229652 9258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:20.229696 9256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:20.229652 9249 server_base.cc:1034] running on GCE node
W20260809 04:40:20.229667 9255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:20.230307 9249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:20.230930 9249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:20.232112 9249 hybrid_clock.cc:648] HybridClock initialized: now 1786250420232074 us; error 50 us; skew 500 ppm
Aug 09 04:40:20 dist-test-slave-vxj2 krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250420, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:20.234965 9249 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:40:20.235882 9249 webserver.cc:460] Webserver started at http://127.5.222.195:42013/ using document root <none> and password file <none>
I20260809 04:40:20.236455 9249 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:20.236550 9249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:20.236824 9249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:20.238898 9249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "f02d7a09391f4e84b30c9c25991f18b0"
format_stamp: "Formatted at 2026-08-09 04:40:20 on dist-test-slave-vxj2"
server_key: "b41a38f4b00d5b112829c4963d2ced56"
server_key_iv: "d5fde62815eb32a4b1584fe4b68be7d9"
server_key_version: "encryptionkey@0"
I20260809 04:40:20.239426 9249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "f02d7a09391f4e84b30c9c25991f18b0"
format_stamp: "Formatted at 2026-08-09 04:40:20 on dist-test-slave-vxj2"
server_key: "b41a38f4b00d5b112829c4963d2ced56"
server_key_iv: "d5fde62815eb32a4b1584fe4b68be7d9"
server_key_version: "encryptionkey@0"
I20260809 04:40:20.243131 9249 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:40:20.245536 9265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:20.246925 9249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:40:20.247018 9249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "f02d7a09391f4e84b30c9c25991f18b0"
format_stamp: "Formatted at 2026-08-09 04:40:20 on dist-test-slave-vxj2"
server_key: "b41a38f4b00d5b112829c4963d2ced56"
server_key_iv: "d5fde62815eb32a4b1584fe4b68be7d9"
server_key_version: "encryptionkey@0"
I20260809 04:40:20.247099 9249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:20.263018 9249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:20.265852 9249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:20.266010 9249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:20.266574 9249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:20.267345 9249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:20.267392 9249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:20.267441 9249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:20.267460 9249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:20.276270 9249 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:46403
I20260809 04:40:20.276309 9391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:46403 every 8 connection(s)
I20260809 04:40:20.277174 9249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:20.283594 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 9249
I20260809 04:40:20.283718 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:40:20.284008 6011 external_mini_cluster.cc:1428] Setting key 9e3012de9a27713b0203eebc1706c77c
Aug 09 04:40:20 dist-test-slave-vxj2 krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250420, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:20.290074 8919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:20.278887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:38221 (local address 127.5.222.254:36427)
0809 04:40:20.279043 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:20.279047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:20.279838 (+ 791us) server_negotiation.cc:407] Connection header received
0809 04:40:20.281100 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:20.281103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:20.281161 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:20.281234 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:20.282307 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:20.282823 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:20.283999 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:20.284288 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:20.284406 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:20.287065 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:20.287068 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:20.287070 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:20.287093 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:20.288304 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:20.288750 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:20.288753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:20.288754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:20.288825 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:20.289105 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:20.289107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:20.289108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:20.289335 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:20.289412 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:20.289850 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 04:40:20.289958 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:40:20.290798 9394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:20.279255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:36427 (local address 127.5.222.195:38221)
0809 04:40:20.279681 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:20.279723 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:20.280908 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:20.281379 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:20.281386 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:20.281729 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:20.282176 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:20.282188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:20.282932 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:20.282934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:20.283872 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:20.283882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:20.284460 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:20.284465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:20.284663 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:20.285278 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:20.285294 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:20.286924 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:20.288412 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:20.288416 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:20.288426 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:20.288667 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:20.288904 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:20.288906 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:20.288907 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:20.289028 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:20.289433 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:20.289437 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:20.289656 (+ 219us) client_negotiation.cc:769] Sending connection context
0809 04:40:20.289804 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 04:40:20.290029 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":114,"threads_started":1}
I20260809 04:40:20.291821 9392 heartbeater.cc:347] Connected to a master server at 127.5.222.254:36427
I20260809 04:40:20.292024 9392 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:20.292450 9392 heartbeater.cc:511] Master 127.5.222.254:36427 requested a full tablet report, sending...
I20260809 04:40:20.293443 8874 ts_manager.cc:194] Registered new tserver with Master: f02d7a09391f4e84b30c9c25991f18b0 (127.5.222.195:46403)
I20260809 04:40:20.293959 8874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:38221
I20260809 04:40:20.297933 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 04:40:20.314495 8919 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:20.312525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.5.222.254:36427)
0809 04:40:20.312701 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:20.312707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:20.313007 (+ 300us) server_negotiation.cc:407] Connection header received
0809 04:40:20.313209 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:20.313214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:20.313274 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:20.313372 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:20.314352 (+ 980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33050: BlockingRecv error: recv got EOF from 127.0.0.1:33050 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:40:20.315780 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 8945
I20260809 04:40:20.331122 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 9096
I20260809 04:40:20.345321 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 9249
I20260809 04:40:20.354431 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 8837
2026-08-09T04:40:20Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2264 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:20 dist-test-slave-vxj2 krb5kdc[9404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:20 dist-test-slave-vxj2 krb5kdc[9404](info): set up 2 sockets
Aug 09 04:40:20 dist-test-slave-vxj2 krb5kdc[9404](info): commencing operation
krb5kdc: starting...
W20260809 04:40:21.465116 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:21 dist-test-slave-vxj2 krb5kdc[9404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:40:21.591739 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:34597
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:34597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:21.701745 9419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:21.702054 9419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:21.702129 9419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:21.706136 9419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:21.706231 9419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:21.706269 9419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:21.706308 9419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:21.706341 9419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:21.711692 9419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:34597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:34597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:21.712795 9419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:21.713883 9419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:21.722834 9424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:21.722748 9425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:21.722803 9419 server_base.cc:1034] running on GCE node
W20260809 04:40:21.722748 9427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:21.723651 9419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:21.724692 9419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:21.725868 9419 hybrid_clock.cc:648] HybridClock initialized: now 1786250421725841 us; error 37 us; skew 500 ppm
Aug 09 04:40:21 dist-test-slave-vxj2 krb5kdc[9404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250421, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:21.733451 9419 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:40:21.734334 9419 webserver.cc:460] Webserver started at http://127.5.222.254:41829/ using document root <none> and password file <none>
I20260809 04:40:21.734867 9419 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:21.734925 9419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:21.735086 9419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:21.737118 9419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "6908524522f94dcfa074680f92806747"
format_stamp: "Formatted at 2026-08-09 04:40:21 on dist-test-slave-vxj2"
server_key: "7edd064d81bed824f344a7338d7baee5"
server_key_iv: "f9f33832c0e2799c9c1894c82914aca8"
server_key_version: "encryptionkey@0"
I20260809 04:40:21.737721 9419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "6908524522f94dcfa074680f92806747"
format_stamp: "Formatted at 2026-08-09 04:40:21 on dist-test-slave-vxj2"
server_key: "7edd064d81bed824f344a7338d7baee5"
server_key_iv: "f9f33832c0e2799c9c1894c82914aca8"
server_key_version: "encryptionkey@0"
I20260809 04:40:21.741582 9419 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:40:21.743935 9435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:21.745199 9419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:21.745337 9419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "6908524522f94dcfa074680f92806747"
format_stamp: "Formatted at 2026-08-09 04:40:21 on dist-test-slave-vxj2"
server_key: "7edd064d81bed824f344a7338d7baee5"
server_key_iv: "f9f33832c0e2799c9c1894c82914aca8"
server_key_version: "encryptionkey@0"
I20260809 04:40:21.745445 9419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:21.760058 9419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:21.763051 9419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:21.763243 9419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:21.770319 9419 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:34597
I20260809 04:40:21.770591 9495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:34597 every 8 connection(s)
I20260809 04:40:21.771237 9419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:21.773962 9497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:21.778373 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 9419
I20260809 04:40:21.778503 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:21.778764 6011 external_mini_cluster.cc:1428] Setting key 54f72c67ab94f20ed96e8d19a75184cf
I20260809 04:40:21.779471 9497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747: Bootstrap starting.
I20260809 04:40:21.782336 9497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:21.783216 9497 log.cc:824] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:21.785310 9497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747: No bootstrap required, opened a new log
Aug 09 04:40:21 dist-test-slave-vxj2 krb5kdc[9404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:21.789515 9497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6908524522f94dcfa074680f92806747" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 34597 } }
I20260809 04:40:21.789840 9497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:21.789902 9497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6908524522f94dcfa074680f92806747, State: Initialized, Role: FOLLOWER
I20260809 04:40:21.790396 9497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6908524522f94dcfa074680f92806747" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 34597 } }
I20260809 04:40:21.790565 9497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:21.790637 9497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:21.790769 9497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:21.791939 9497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6908524522f94dcfa074680f92806747" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 34597 } }
I20260809 04:40:21.792291 9497 leader_election.cc:302] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6908524522f94dcfa074680f92806747; no voters:
I20260809 04:40:21.792635 9497 leader_election.cc:288] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:21.792827 9502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:21.793850 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:21.780012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39946 (local address 127.5.222.254:34597)
0809 04:40:21.780433 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:21.780457 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:21.780497 (+ 40us) server_negotiation.cc:407] Connection header received
0809 04:40:21.781376 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:21.781399 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:21.781805 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:21.782147 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:21.782938 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:21.784281 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:21.785130 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:21.785843 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:21.786053 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:21.788480 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:21.788486 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:21.788504 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:21.788538 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:21.790605 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:21.791128 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:21.791134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:21.791139 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:21.791284 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:21.791545 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:21.791548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:21.791550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:21.792083 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:21.792281 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:21.792489 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 04:40:21.792796 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":102,"threads_started":1}
I20260809 04:40:21.793972 9497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:21.793092 9502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [term 1 LEADER]: Becoming Leader. State: Replica: 6908524522f94dcfa074680f92806747, State: Running, Role: LEADER
I20260809 04:40:21.795194 9502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6908524522f94dcfa074680f92806747" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 34597 } }
I20260809 04:40:21.796784 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:40:21.797014 9503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6908524522f94dcfa074680f92806747. Latest consensus state: current_term: 1 leader_uuid: "6908524522f94dcfa074680f92806747" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6908524522f94dcfa074680f92806747" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 34597 } } }
I20260809 04:40:21.797060 9504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6908524522f94dcfa074680f92806747" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6908524522f94dcfa074680f92806747" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 34597 } } }
I20260809 04:40:21.797137 9503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:21.797178 9504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:21.797541 9518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:21.800347 9518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:21.806874 9518 catalog_manager.cc:1347] Generated new cluster ID: 3bd094ce3281442593e860d62b862bd5
I20260809 04:40:21.806941 9518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:21.815317 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:21.810370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39948 (local address 127.5.222.254:34597)
0809 04:40:21.810490 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:21.810495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:21.810881 (+ 386us) server_negotiation.cc:407] Connection header received
0809 04:40:21.811053 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:21.811056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:21.811109 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:21.811192 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:21.811709 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:21.812252 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:21.813114 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:21.813434 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:21.813552 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:21.814411 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:21.814423 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:21.814426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:21.814842 (+ 416us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:21.814870 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:21.814885 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:21.814989 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:21.815101 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 04:40:21.815166 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":43}
I20260809 04:40:21.838359 9518 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:21.839624 9518 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:21.851089 9518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6908524522f94dcfa074680f92806747: Generated new TSK 0
I20260809 04:40:21.851761 9518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:40:21.860606 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 9419
2026-08-09T04:40:21Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1504 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:21 dist-test-slave-vxj2 krb5kdc[9535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:21 dist-test-slave-vxj2 krb5kdc[9535](info): set up 2 sockets
Aug 09 04:40:21 dist-test-slave-vxj2 krb5kdc[9535](info): commencing operation
krb5kdc: starting...
W20260809 04:40:22.951900 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:23 dist-test-slave-vxj2 krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:40:23.075580 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:39975
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:39975
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:23.186821 9549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:23.187126 9549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:23.187197 9549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:23.191253 9549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:23.191346 9549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:23.191385 9549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:23.191445 9549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:23.191486 9549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:23.197387 9549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:39975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:39975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:23.198516 9549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:23.199608 9549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:23.208815 9556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:23.208994 9549 server_base.cc:1034] running on GCE node
W20260809 04:40:23.208866 9558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:23.208820 9554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:23.209780 9549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:23.210898 9549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:23.212088 9549 hybrid_clock.cc:648] HybridClock initialized: now 1786250423212070 us; error 25 us; skew 500 ppm
Aug 09 04:40:23 dist-test-slave-vxj2 krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250423, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:23.214838 9549 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:40:23.215747 9549 webserver.cc:460] Webserver started at http://127.5.222.254:42005/ using document root <none> and password file <none>
I20260809 04:40:23.216261 9549 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:23.216360 9549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:23.216529 9549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:23.218578 9549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "299e9ace4d024143bfbbb59f33e8427e"
format_stamp: "Formatted at 2026-08-09 04:40:23 on dist-test-slave-vxj2"
server_key: "23471bef6ca1c078a3023ceefe648a76"
server_key_iv: "9f368ba02ffe42c67d106566047d5eb5"
server_key_version: "encryptionkey@0"
I20260809 04:40:23.219074 9549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "299e9ace4d024143bfbbb59f33e8427e"
format_stamp: "Formatted at 2026-08-09 04:40:23 on dist-test-slave-vxj2"
server_key: "23471bef6ca1c078a3023ceefe648a76"
server_key_iv: "9f368ba02ffe42c67d106566047d5eb5"
server_key_version: "encryptionkey@0"
I20260809 04:40:23.222764 9549 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 04:40:23.225092 9565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:23.226356 9549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:40:23.226469 9549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "299e9ace4d024143bfbbb59f33e8427e"
format_stamp: "Formatted at 2026-08-09 04:40:23 on dist-test-slave-vxj2"
server_key: "23471bef6ca1c078a3023ceefe648a76"
server_key_iv: "9f368ba02ffe42c67d106566047d5eb5"
server_key_version: "encryptionkey@0"
I20260809 04:40:23.226557 9549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:23.250977 9549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:23.253791 9549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:23.253947 9549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:23.260913 9549 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:39975
I20260809 04:40:23.261236 9625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:39975 every 8 connection(s)
I20260809 04:40:23.261986 9549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:23.264657 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:23.270064 9626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e: Bootstrap starting.
I20260809 04:40:23.272104 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 9549
I20260809 04:40:23.272198 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:23.272500 6011 external_mini_cluster.cc:1428] Setting key 096d31c5468bea52892816c4d44ea05c
I20260809 04:40:23.272517 9626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:23.273440 9626 log.cc:824] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:23.275388 9626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e: No bootstrap required, opened a new log
I20260809 04:40:23.279518 9626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299e9ace4d024143bfbbb59f33e8427e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39975 } }
I20260809 04:40:23.279781 9626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:23.279870 9626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 299e9ace4d024143bfbbb59f33e8427e, State: Initialized, Role: FOLLOWER
I20260809 04:40:23.280396 9626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299e9ace4d024143bfbbb59f33e8427e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39975 } }
I20260809 04:40:23.280582 9626 raft_consensus.cc:388] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:23.280650 9626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:23.280748 9626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [term 0 FOLLOWER]: Advancing to term 1
Aug 09 04:40:23 dist-test-slave-vxj2 krb5kdc[9535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:23.281813 9626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299e9ace4d024143bfbbb59f33e8427e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39975 } }
I20260809 04:40:23.282168 9626 leader_election.cc:302] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 299e9ace4d024143bfbbb59f33e8427e; no voters:
I20260809 04:40:23.282531 9626 leader_election.cc:288] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:23.282630 9631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:23.282953 9631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [term 1 LEADER]: Becoming Leader. State: Replica: 299e9ace4d024143bfbbb59f33e8427e, State: Running, Role: LEADER
I20260809 04:40:23.283314 9631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299e9ace4d024143bfbbb59f33e8427e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39975 } }
I20260809 04:40:23.283937 9626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:23.284909 9632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "299e9ace4d024143bfbbb59f33e8427e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299e9ace4d024143bfbbb59f33e8427e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39975 } } }
I20260809 04:40:23.285025 9632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:23.285421 9633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 299e9ace4d024143bfbbb59f33e8427e. Latest consensus state: current_term: 1 leader_uuid: "299e9ace4d024143bfbbb59f33e8427e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299e9ace4d024143bfbbb59f33e8427e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39975 } } }
I20260809 04:40:23.285560 9633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:23.285854 9642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:23.288139 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:23.273685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56848 (local address 127.5.222.254:39975)
0809 04:40:23.274145 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:23.274170 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:23.274204 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:40:23.274814 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:23.274829 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:23.275129 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:23.275398 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:23.276310 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:23.277375 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:23.278640 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:23.279082 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:23.279194 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:23.282035 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:23.282041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:23.282057 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:23.282088 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:23.284205 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:23.284820 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:23.284827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:23.284834 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:23.284994 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:23.285378 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:23.285381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:23.285383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:23.285945 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:23.286757 (+ 812us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:23.286926 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 04:40:23.287201 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":172,"threads_started":1}
I20260809 04:40:23.288647 9642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:23.294893 9642 catalog_manager.cc:1347] Generated new cluster ID: 8389fa0d61594da0a1a13105c1e9d7bb
I20260809 04:40:23.294992 9642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:23.320735 9642 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:23.321779 9642 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:23.337839 9642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 299e9ace4d024143bfbbb59f33e8427e: Generated new TSK 0
I20260809 04:40:23.338681 9642 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:40:23.372179 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:39975
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:23.489298 9654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:23.489586 9654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:23.489657 9654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:23.493810 9654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:23.493894 9654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:23.493983 9654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:23.499267 9654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:39975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:23.500365 9654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:23.501757 9654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:23.511538 9660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:23.511529 9659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:23.511701 9654 server_base.cc:1034] running on GCE node
W20260809 04:40:23.511744 9662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:23.512221 9654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:23.512920 9654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:23.514108 9654 hybrid_clock.cc:648] HybridClock initialized: now 1786250423514076 us; error 45 us; skew 500 ppm
Aug 09 04:40:23 dist-test-slave-vxj2 krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250423, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:23.516791 9654 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:40:23.517711 9654 webserver.cc:460] Webserver started at http://127.5.222.193:34241/ using document root <none> and password file <none>
I20260809 04:40:23.518362 9654 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:23.518466 9654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:23.518682 9654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:23.520788 9654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "93ee0cb6e12e4bbebb90eec8f4c67f6f"
format_stamp: "Formatted at 2026-08-09 04:40:23 on dist-test-slave-vxj2"
server_key: "d29fcd1cc906b36362c13cd38c9debd4"
server_key_iv: "ce9ebac2fa6edcf4e9ac7a119f54fded"
server_key_version: "encryptionkey@0"
I20260809 04:40:23.521410 9654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "93ee0cb6e12e4bbebb90eec8f4c67f6f"
format_stamp: "Formatted at 2026-08-09 04:40:23 on dist-test-slave-vxj2"
server_key: "d29fcd1cc906b36362c13cd38c9debd4"
server_key_iv: "ce9ebac2fa6edcf4e9ac7a119f54fded"
server_key_version: "encryptionkey@0"
I20260809 04:40:23.525048 9654 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:23.527415 9669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:23.528692 9654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:23.528822 9654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "93ee0cb6e12e4bbebb90eec8f4c67f6f"
format_stamp: "Formatted at 2026-08-09 04:40:23 on dist-test-slave-vxj2"
server_key: "d29fcd1cc906b36362c13cd38c9debd4"
server_key_iv: "ce9ebac2fa6edcf4e9ac7a119f54fded"
server_key_version: "encryptionkey@0"
I20260809 04:40:23.528929 9654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:23.546154 9654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:23.549050 9654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:23.549275 9654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:23.549880 9654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:23.550694 9654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:23.550765 9654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:23.550837 9654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:23.550889 9654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:23.559756 9654 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:41259
I20260809 04:40:23.559789 9802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:41259 every 8 connection(s)
I20260809 04:40:23.560628 9654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:23.569221 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 9654
I20260809 04:40:23.569361 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:23.569653 6011 external_mini_cluster.cc:1428] Setting key f8b5e736e32c994948eb16f9a6b7c1fe
Aug 09 04:40:23 dist-test-slave-vxj2 krb5kdc[9535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250423, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:23.574810 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:23.562460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:39601 (local address 127.5.222.254:39975)
0809 04:40:23.562643 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:23.562648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:23.563459 (+ 811us) server_negotiation.cc:407] Connection header received
0809 04:40:23.564773 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:23.564775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:23.564822 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:23.564897 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:23.566128 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:23.566676 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:23.567804 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:23.568021 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:23.568096 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:23.570894 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:23.570901 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:23.570905 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:23.570931 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:23.572401 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:23.573073 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:23.573078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:23.573083 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:23.573219 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:23.573578 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:23.573582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:23.573585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:23.573819 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:23.573946 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:23.574418 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 04:40:23.574559 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 04:40:23.575444 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:23.562811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:39975 (local address 127.5.222.193:39601)
0809 04:40:23.563293 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:23.563337 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:23.564582 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:23.565055 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:23.565061 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:23.565520 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:23.565995 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:23.566007 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:23.566809 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:23.566812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:23.567695 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:23.567704 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:23.568149 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:23.568151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:23.568313 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:23.568956 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:23.568973 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:23.570740 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:23.572529 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:23.572539 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:23.572555 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:23.572955 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:23.573324 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:23.573327 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:23.573328 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:23.573481 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:23.573926 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:23.573931 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:23.574176 (+ 245us) client_negotiation.cc:769] Sending connection context
0809 04:40:23.574332 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 04:40:23.574576 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
I20260809 04:40:23.576916 9803 heartbeater.cc:347] Connected to a master server at 127.5.222.254:39975
I20260809 04:40:23.577236 9803 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:23.577878 9803 heartbeater.cc:511] Master 127.5.222.254:39975 requested a full tablet report, sending...
I20260809 04:40:23.579440 9583 ts_manager.cc:194] Registered new tserver with Master: 93ee0cb6e12e4bbebb90eec8f4c67f6f (127.5.222.193:41259)
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
I20260809 04:40:23.580583 9583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:39601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:40:23.598172 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:39975
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:23.709563 9808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:23.709803 9808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:23.709843 9808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:23.713898 9808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:23.713950 9808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:23.714004 9808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:23.719111 9808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:39975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:23.720098 9808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:23.721092 9808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:23.731438 9813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:23.731406 9816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:23.731431 9814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:23.731633 9808 server_base.cc:1034] running on GCE node
I20260809 04:40:23.732136 9808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:23.732831 9808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:23.734010 9808 hybrid_clock.cc:648] HybridClock initialized: now 1786250423733984 us; error 39 us; skew 500 ppm
Aug 09 04:40:23 dist-test-slave-vxj2 krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250423, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:23.736764 9808 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:40:23.737681 9808 webserver.cc:460] Webserver started at http://127.5.222.194:40227/ using document root <none> and password file <none>
I20260809 04:40:23.738222 9808 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:23.738314 9808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:23.738552 9808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:23.740486 9808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "624850caa9134057b66b20cb6a42efa9"
format_stamp: "Formatted at 2026-08-09 04:40:23 on dist-test-slave-vxj2"
server_key: "bf24dc0379dedd4ce87078772b0995ee"
server_key_iv: "d0fc4c21ec86eaeec52dbb7fdad34374"
server_key_version: "encryptionkey@0"
I20260809 04:40:23.741034 9808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "624850caa9134057b66b20cb6a42efa9"
format_stamp: "Formatted at 2026-08-09 04:40:23 on dist-test-slave-vxj2"
server_key: "bf24dc0379dedd4ce87078772b0995ee"
server_key_iv: "d0fc4c21ec86eaeec52dbb7fdad34374"
server_key_version: "encryptionkey@0"
I20260809 04:40:23.744793 9808 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:23.747154 9824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:23.748344 9808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:23.748492 9808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "624850caa9134057b66b20cb6a42efa9"
format_stamp: "Formatted at 2026-08-09 04:40:23 on dist-test-slave-vxj2"
server_key: "bf24dc0379dedd4ce87078772b0995ee"
server_key_iv: "d0fc4c21ec86eaeec52dbb7fdad34374"
server_key_version: "encryptionkey@0"
I20260809 04:40:23.748600 9808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:23.778013 9808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:23.781388 9808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:23.781603 9808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:23.782254 9808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:23.783152 9808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:23.783221 9808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:23.783291 9808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:23.783334 9808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:23.792683 9808 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:38937
I20260809 04:40:23.792708 9949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:38937 every 8 connection(s)
I20260809 04:40:23.793628 9808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:23.794775 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 9808
I20260809 04:40:23.794910 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:23.795209 6011 external_mini_cluster.cc:1428] Setting key 950ef62953f4f766c25a525d0123bfc4
Aug 09 04:40:23 dist-test-slave-vxj2 krb5kdc[9535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250423, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
I20260809 04:40:23.809460 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:23.795548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:35083 (local address 127.5.222.254:39975)
0809 04:40:23.795691 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:23.795694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:23.796671 (+ 977us) server_negotiation.cc:407] Connection header received
0809 04:40:23.798162 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:23.798166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:23.798218 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:23.798297 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:23.799454 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:23.799980 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:23.801869 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:23.802092 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:23.802188 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:23.805346 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:23.805351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:23.805356 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:23.805390 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:23.806991 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:23.807596 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:23.807598 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:23.807600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:23.807676 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:23.808100 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:23.808104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:23.808105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:23.808376 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:23.808513 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:23.809213 (+ 700us) server_negotiation.cc:299] Negotiation successful
0809 04:40:23.809320 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:40:23.810688 9952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:23.795994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:39975 (local address 127.5.222.194:35083)
0809 04:40:23.796493 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:23.796567 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:23.797958 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:23.798463 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:23.798470 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:23.798830 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:23.799287 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:23.799300 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:23.800122 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:23.800128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:23.801720 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:23.801735 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:23.802264 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:23.802270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:23.802528 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:23.803172 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:23.803195 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:23.805065 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:23.807109 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:23.807115 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:23.807128 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:23.807468 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:23.807802 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:23.807804 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:23.807806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:23.807980 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:23.808524 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:23.808532 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:23.808885 (+ 353us) client_negotiation.cc:769] Sending connection context
0809 04:40:23.809116 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 04:40:23.809578 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":107,"threads_started":1}
I20260809 04:40:23.811970 9950 heartbeater.cc:347] Connected to a master server at 127.5.222.254:39975
I20260809 04:40:23.812253 9950 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:23.812793 9950 heartbeater.cc:511] Master 127.5.222.254:39975 requested a full tablet report, sending...
I20260809 04:40:23.813958 9583 ts_manager.cc:194] Registered new tserver with Master: 624850caa9134057b66b20cb6a42efa9 (127.5.222.194:38937)
I20260809 04:40:23.814586 9583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:35083
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:40:23.824311 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:39975
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:23.934984 9955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:23.935242 9955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:23.935285 9955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:23.939254 9955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:23.939311 9955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:23.939368 9955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:23.944697 9955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:39975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:23.945745 9955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:23.946707 9955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:23.956538 9962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:23.956516 9963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:23.956557 9965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:23.956840 9955 server_base.cc:1034] running on GCE node
I20260809 04:40:23.957262 9955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:23.957881 9955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:23.959062 9955 hybrid_clock.cc:648] HybridClock initialized: now 1786250423959054 us; error 31 us; skew 500 ppm
Aug 09 04:40:23 dist-test-slave-vxj2 krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250423, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:23.961786 9955 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:40:23.962574 9955 webserver.cc:460] Webserver started at http://127.5.222.195:34531/ using document root <none> and password file <none>
I20260809 04:40:23.963102 9955 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:23.963160 9955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:23.963318 9955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:23.965308 9955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "6a9a9a1f22094049874e7b8eecacef99"
format_stamp: "Formatted at 2026-08-09 04:40:23 on dist-test-slave-vxj2"
server_key: "2fb029d7e9da00e985cb98594bcaa2c0"
server_key_iv: "101fc39700b641a5348269f2bb77a309"
server_key_version: "encryptionkey@0"
I20260809 04:40:23.965850 9955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "6a9a9a1f22094049874e7b8eecacef99"
format_stamp: "Formatted at 2026-08-09 04:40:23 on dist-test-slave-vxj2"
server_key: "2fb029d7e9da00e985cb98594bcaa2c0"
server_key_iv: "101fc39700b641a5348269f2bb77a309"
server_key_version: "encryptionkey@0"
I20260809 04:40:23.969468 9955 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:40:23.971745 9973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:23.973027 9955 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:23.973132 9955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "6a9a9a1f22094049874e7b8eecacef99"
format_stamp: "Formatted at 2026-08-09 04:40:23 on dist-test-slave-vxj2"
server_key: "2fb029d7e9da00e985cb98594bcaa2c0"
server_key_iv: "101fc39700b641a5348269f2bb77a309"
server_key_version: "encryptionkey@0"
I20260809 04:40:23.973273 9955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:23.982069 9955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:23.985186 9955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:23.985394 9955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:23.986060 9955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:23.986937 9955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:23.986985 9955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:23.987057 9955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:23.987097 9955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:23.995877 9955 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:40973
I20260809 04:40:23.995898 10106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:40973 every 8 connection(s)
I20260809 04:40:23.996762 9955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:24.000716 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 9955
I20260809 04:40:24.000872 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:40:24.001227 6011 external_mini_cluster.cc:1428] Setting key 059a03fdc3f02ac3afe1b27361e088ea
Aug 09 04:40:24 dist-test-slave-vxj2 krb5kdc[9535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250423, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:24.010881 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:23.998618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:44439 (local address 127.5.222.254:39975)
0809 04:40:23.998765 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:23.998769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:23.999572 (+ 803us) server_negotiation.cc:407] Connection header received
0809 04:40:24.000995 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:24.000999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:24.001066 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:24.001191 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:24.002658 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:24.003227 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:24.004395 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:24.004612 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:24.004691 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:24.007443 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:24.007446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:24.007448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:24.007471 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:24.008883 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:24.009506 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:24.009508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:24.009510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:24.009585 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:24.009909 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:24.009911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:24.009913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:24.010089 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:24.010161 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:24.010657 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 04:40:24.010755 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:40:24.011811 10111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:23.998983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:39975 (local address 127.5.222.195:44439)
0809 04:40:23.999423 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:23.999467 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:24.000748 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:24.001394 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:24.001408 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:24.001889 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:24.002501 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:24.002523 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:24.003340 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:24.003342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:24.004282 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:24.004294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:24.004739 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:24.004741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:24.004912 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:24.005537 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:24.005554 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:24.007282 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:24.009008 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:24.009014 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:24.009026 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:24.009407 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:24.009666 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:24.009669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:24.009670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:24.009821 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:24.010177 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:24.010182 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:24.010423 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 04:40:24.010600 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 04:40:24.010915 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":108,"threads_started":1}
I20260809 04:40:24.012946 10109 heartbeater.cc:347] Connected to a master server at 127.5.222.254:39975
I20260809 04:40:24.013242 10109 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:24.013768 10109 heartbeater.cc:511] Master 127.5.222.254:39975 requested a full tablet report, sending...
I20260809 04:40:24.014986 9583 ts_manager.cc:194] Registered new tserver with Master: 6a9a9a1f22094049874e7b8eecacef99 (127.5.222.195:40973)
I20260809 04:40:24.015399 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:40:24.015744 9583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:44439
Aug 09 04:40:24 dist-test-slave-vxj2 krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250424, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 04:40:24 dist-test-slave-vxj2 krb5kdc[9535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250424, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:24.043041 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:24.034720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56850 (local address 127.5.222.254:39975)
0809 04:40:24.034864 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:24.034868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:24.034966 (+ 98us) server_negotiation.cc:407] Connection header received
0809 04:40:24.035089 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:24.035092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:24.035140 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:24.035212 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:24.035897 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:24.036449 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:24.037333 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:24.037554 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:24.037647 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:24.040006 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:24.040009 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:24.040011 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:24.040049 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:24.041582 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:24.041974 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:24.041976 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:24.041978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:24.042063 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:24.042294 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:24.042296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:24.042297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:24.042514 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:24.042628 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:24.042813 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 04:40:24.042912 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 04:40:24.043479 9583 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56850
I20260809 04:40:24.044415 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 9654
I20260809 04:40:24.052677 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 9808
I20260809 04:40:24.058755 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 9955
I20260809 04:40:24.066298 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 9549
2026-08-09T04:40:24Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2207 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:24 dist-test-slave-vxj2 krb5kdc[10122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:24 dist-test-slave-vxj2 krb5kdc[10122](info): set up 2 sockets
Aug 09 04:40:24 dist-test-slave-vxj2 krb5kdc[10122](info): commencing operation
krb5kdc: starting...
W20260809 04:40:25.272374 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:25 dist-test-slave-vxj2 krb5kdc[10122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:40:25.394548 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:45781
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:45781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:25.504640 10139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:25.504897 10139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:25.504940 10139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:25.508930 10139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:25.508993 10139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:25.509012 10139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:25.509029 10139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:25.509044 10139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:25.514245 10139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:45781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:45781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:25.515414 10139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:25.516469 10139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:25.525125 10147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:25.525136 10144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:25.525172 10145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:25.525491 10139 server_base.cc:1034] running on GCE node
I20260809 04:40:25.526028 10139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:25.527125 10139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:25.528328 10139 hybrid_clock.cc:648] HybridClock initialized: now 1786250425528293 us; error 52 us; skew 500 ppm
Aug 09 04:40:25 dist-test-slave-vxj2 krb5kdc[10122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250425, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:25.531272 10139 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:40:25.532222 10139 webserver.cc:460] Webserver started at http://127.5.222.254:43257/ using document root <none> and password file <none>
I20260809 04:40:25.532793 10139 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:25.532886 10139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:25.533136 10139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:25.535218 10139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "08ee43256b8a48d9b08b849ae17aff7d"
format_stamp: "Formatted at 2026-08-09 04:40:25 on dist-test-slave-vxj2"
server_key: "f262a7bdce2b9d7a3ebdfba0f0f6ec99"
server_key_iv: "fdacff1f11481882a154babb4d06c6d6"
server_key_version: "encryptionkey@0"
I20260809 04:40:25.535842 10139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "08ee43256b8a48d9b08b849ae17aff7d"
format_stamp: "Formatted at 2026-08-09 04:40:25 on dist-test-slave-vxj2"
server_key: "f262a7bdce2b9d7a3ebdfba0f0f6ec99"
server_key_iv: "fdacff1f11481882a154babb4d06c6d6"
server_key_version: "encryptionkey@0"
I20260809 04:40:25.539505 10139 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260809 04:40:25.541917 10155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:25.543118 10139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260809 04:40:25.543257 10139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "08ee43256b8a48d9b08b849ae17aff7d"
format_stamp: "Formatted at 2026-08-09 04:40:25 on dist-test-slave-vxj2"
server_key: "f262a7bdce2b9d7a3ebdfba0f0f6ec99"
server_key_iv: "fdacff1f11481882a154babb4d06c6d6"
server_key_version: "encryptionkey@0"
I20260809 04:40:25.543365 10139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:25.560474 10139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:25.563395 10139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:25.563599 10139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:25.571167 10139 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:45781
I20260809 04:40:25.571195 10215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:45781 every 8 connection(s)
I20260809 04:40:25.572104 10139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:25.575027 10216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:25.580927 10216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d: Bootstrap starting.
I20260809 04:40:25.580945 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 10139
I20260809 04:40:25.581081 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:25.581480 6011 external_mini_cluster.cc:1428] Setting key d8488d97e401b7501497d18adadcc6b3
I20260809 04:40:25.583859 10216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:25.584666 10216 log.cc:824] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:25.586725 10216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d: No bootstrap required, opened a new log
Aug 09 04:40:25 dist-test-slave-vxj2 krb5kdc[10122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:25.591287 10216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ee43256b8a48d9b08b849ae17aff7d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45781 } }
I20260809 04:40:25.591588 10216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:25.591660 10216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08ee43256b8a48d9b08b849ae17aff7d, State: Initialized, Role: FOLLOWER
I20260809 04:40:25.592150 10216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ee43256b8a48d9b08b849ae17aff7d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45781 } }
I20260809 04:40:25.592360 10216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:25.592482 10216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:25.592634 10216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:25.593809 10216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ee43256b8a48d9b08b849ae17aff7d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45781 } }
I20260809 04:40:25.594224 10216 leader_election.cc:302] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08ee43256b8a48d9b08b849ae17aff7d; no voters:
I20260809 04:40:25.594643 10216 leader_election.cc:288] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:25.594718 10221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:25.594982 10221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [term 1 LEADER]: Becoming Leader. State: Replica: 08ee43256b8a48d9b08b849ae17aff7d, State: Running, Role: LEADER
I20260809 04:40:25.595459 10221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ee43256b8a48d9b08b849ae17aff7d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45781 } }
I20260809 04:40:25.596056 10216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:25.596793 10221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08ee43256b8a48d9b08b849ae17aff7d. Latest consensus state: current_term: 1 leader_uuid: "08ee43256b8a48d9b08b849ae17aff7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ee43256b8a48d9b08b849ae17aff7d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45781 } } }
I20260809 04:40:25.597039 10221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:25.597101 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:25.582945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49768 (local address 127.5.222.254:45781)
0809 04:40:25.583382 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:25.583397 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:25.583450 (+ 53us) server_negotiation.cc:407] Connection header received
0809 04:40:25.584136 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:25.584155 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:25.584482 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:25.584812 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:25.585668 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:25.586643 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:25.587554 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:25.588024 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:25.588199 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:25.591180 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:25.591185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:25.591197 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:25.591223 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:25.593308 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:25.593882 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:25.593886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:25.593890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:25.594061 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:25.594414 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:25.594417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:25.594419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:25.594988 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:25.595776 (+ 788us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:25.595894 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 04:40:25.596152 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":133,"threads_started":1}
I20260809 04:40:25.597224 10222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08ee43256b8a48d9b08b849ae17aff7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ee43256b8a48d9b08b849ae17aff7d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45781 } } }
I20260809 04:40:25.597332 10222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d [sys.catalog]: This master's current role is: LEADER
W20260809 04:40:25.601285 10235 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:40:25.601366 10235 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:40:25.601447 10237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:25.602308 10237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:25.608048 10237 catalog_manager.cc:1347] Generated new cluster ID: 14ffb39e8e434b6ba0ec389c71381ebf
I20260809 04:40:25.608129 10237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:25.629537 10237 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:25.630679 10237 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:25.647960 10237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08ee43256b8a48d9b08b849ae17aff7d: Generated new TSK 0
I20260809 04:40:25.648720 10237 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:40:25.686636 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:45781
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:25.826076 10242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:25.826352 10242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:25.826398 10242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:25.830492 10242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:25.830562 10242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:25.830624 10242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:25.835875 10242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:45781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:25.836869 10242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:25.837873 10242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:25.847572 10249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:25.847564 10251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:25.847563 10248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:25.847945 10242 server_base.cc:1034] running on GCE node
I20260809 04:40:25.848340 10242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:25.848959 10242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:25.850114 10242 hybrid_clock.cc:648] HybridClock initialized: now 1786250425850084 us; error 72 us; skew 500 ppm
Aug 09 04:40:25 dist-test-slave-vxj2 krb5kdc[10122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250425, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:25.852845 10242 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:40:25.853706 10242 webserver.cc:460] Webserver started at http://127.5.222.193:36887/ using document root <none> and password file <none>
I20260809 04:40:25.854245 10242 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:25.854306 10242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:25.854513 10242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:25.856483 10242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "78ed3637b3e8445386cd43c79b498b89"
format_stamp: "Formatted at 2026-08-09 04:40:25 on dist-test-slave-vxj2"
server_key: "1f12c3cd1c0a60faeb11fb74f6f817ab"
server_key_iv: "baaeb5bd85ccd3ed90ecbb229590ba0c"
server_key_version: "encryptionkey@0"
I20260809 04:40:25.857093 10242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "78ed3637b3e8445386cd43c79b498b89"
format_stamp: "Formatted at 2026-08-09 04:40:25 on dist-test-slave-vxj2"
server_key: "1f12c3cd1c0a60faeb11fb74f6f817ab"
server_key_iv: "baaeb5bd85ccd3ed90ecbb229590ba0c"
server_key_version: "encryptionkey@0"
I20260809 04:40:25.860594 10242 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:40:25.863065 10260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:25.864275 10242 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:40:25.864409 10242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "78ed3637b3e8445386cd43c79b498b89"
format_stamp: "Formatted at 2026-08-09 04:40:25 on dist-test-slave-vxj2"
server_key: "1f12c3cd1c0a60faeb11fb74f6f817ab"
server_key_iv: "baaeb5bd85ccd3ed90ecbb229590ba0c"
server_key_version: "encryptionkey@0"
I20260809 04:40:25.864517 10242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:25.875172 10242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:25.878024 10242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:25.878218 10242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:25.878772 10242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:25.879561 10242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:25.879632 10242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:25.879702 10242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:25.879755 10242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:25.889429 10242 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:38393
I20260809 04:40:25.889474 10387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:38393 every 8 connection(s)
I20260809 04:40:25.890282 10242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:25.894332 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 10242
I20260809 04:40:25.894490 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:25.894809 6011 external_mini_cluster.cc:1428] Setting key 3538e9e736204ad0c13bd15edcd23d81
Aug 09 04:40:25 dist-test-slave-vxj2 krb5kdc[10122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250425, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:25.905435 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:25.892224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:36963 (local address 127.5.222.254:45781)
0809 04:40:25.892365 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:25.892369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:25.893285 (+ 916us) server_negotiation.cc:407] Connection header received
0809 04:40:25.894736 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:25.894740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:25.894798 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:25.894877 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:25.896099 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:25.896691 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:25.897961 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:25.898359 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:25.898516 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:25.901453 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:25.901460 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:25.901463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:25.901492 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:25.902998 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:25.903735 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:25.903737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:25.903738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:25.903811 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:25.904235 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:25.904237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:25.904238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:25.904404 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:25.904479 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:25.905200 (+ 721us) server_negotiation.cc:299] Negotiation successful
0809 04:40:25.905313 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
I20260809 04:40:25.906697 10390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:25.892628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:45781 (local address 127.5.222.193:36963)
0809 04:40:25.893091 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:25.893133 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:25.894482 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:25.895049 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:25.895056 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:25.895383 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:25.895955 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:25.895969 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:25.896841 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:25.896844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:25.897833 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:25.897843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:25.898557 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:25.898560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:25.898722 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:25.899413 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:25.899430 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:25.901281 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:25.903128 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:25.903136 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:25.903155 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:25.903623 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:25.903909 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:25.903913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:25.903915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:25.904127 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:25.904521 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:25.904528 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:25.904872 (+ 344us) client_negotiation.cc:769] Sending connection context
0809 04:40:25.905119 (+ 247us) client_negotiation.cc:240] Negotiation successful
0809 04:40:25.905568 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":123,"threads_started":1}
I20260809 04:40:25.907994 10388 heartbeater.cc:347] Connected to a master server at 127.5.222.254:45781
I20260809 04:40:25.908236 10388 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:25.908805 10388 heartbeater.cc:511] Master 127.5.222.254:45781 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:40:25.910540 10174 ts_manager.cc:194] Registered new tserver with Master: 78ed3637b3e8445386cd43c79b498b89 (127.5.222.193:38393)
I20260809 04:40:25.911944 10174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:36963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:40:25.926460 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:45781
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:26.036259 10394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:26.036511 10394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:26.036556 10394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:26.040632 10394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:26.040689 10394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:26.040745 10394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:26.045905 10394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:45781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:26.046957 10394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:26.048005 10394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:40:26.058027 10394 server_base.cc:1034] running on GCE node
W20260809 04:40:26.057914 10407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:26.057914 10402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:26.057940 10401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:26.058660 10394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:26.059283 10394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:26.060461 10394 hybrid_clock.cc:648] HybridClock initialized: now 1786250426060444 us; error 25 us; skew 500 ppm
Aug 09 04:40:26 dist-test-slave-vxj2 krb5kdc[10122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250426, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:26.063300 10394 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:40:26.064146 10394 webserver.cc:460] Webserver started at http://127.5.222.194:44445/ using document root <none> and password file <none>
I20260809 04:40:26.064710 10394 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:26.064807 10394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:26.065052 10394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:26.067271 10394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "2ff54dd5e1fa482d89cf3b46d79b53e9"
format_stamp: "Formatted at 2026-08-09 04:40:26 on dist-test-slave-vxj2"
server_key: "b0fa370a5a3a3459ab38659e561679d9"
server_key_iv: "44438198fe7acf213eb72d4c1427687c"
server_key_version: "encryptionkey@0"
I20260809 04:40:26.067871 10394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "2ff54dd5e1fa482d89cf3b46d79b53e9"
format_stamp: "Formatted at 2026-08-09 04:40:26 on dist-test-slave-vxj2"
server_key: "b0fa370a5a3a3459ab38659e561679d9"
server_key_iv: "44438198fe7acf213eb72d4c1427687c"
server_key_version: "encryptionkey@0"
I20260809 04:40:26.071692 10394 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:40:26.074224 10415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:26.075410 10394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:26.075557 10394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "2ff54dd5e1fa482d89cf3b46d79b53e9"
format_stamp: "Formatted at 2026-08-09 04:40:26 on dist-test-slave-vxj2"
server_key: "b0fa370a5a3a3459ab38659e561679d9"
server_key_iv: "44438198fe7acf213eb72d4c1427687c"
server_key_version: "encryptionkey@0"
I20260809 04:40:26.075670 10394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:26.093041 10394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:26.095803 10394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:26.096004 10394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:26.096626 10394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:26.097534 10394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:26.097604 10394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:26.097676 10394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:26.097738 10394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:26.106705 10394 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:38879
I20260809 04:40:26.106726 10543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:38879 every 8 connection(s)
I20260809 04:40:26.107571 10394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:26.112824 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 10394
I20260809 04:40:26.112957 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:26.113291 6011 external_mini_cluster.cc:1428] Setting key 9ad01d2070101e7381124fb47c3c53f3
Aug 09 04:40:26 dist-test-slave-vxj2 krb5kdc[10122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250426, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:26.122609 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:26.109402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:46461 (local address 127.5.222.254:45781)
0809 04:40:26.109569 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:26.109574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:26.110390 (+ 816us) server_negotiation.cc:407] Connection header received
0809 04:40:26.111687 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:26.111690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:26.111737 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:26.111812 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:26.112939 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:26.113712 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:26.115212 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:26.115598 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:26.115725 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:26.118570 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:26.118577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:26.118583 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:26.118616 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:26.120374 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:26.120865 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:26.120873 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:26.120876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:26.120981 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:26.121383 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:26.121390 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:26.121395 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:26.121692 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:26.121819 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:26.122291 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 04:40:26.122440 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:40:26.123413 10546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:26.109788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:45781 (local address 127.5.222.194:46461)
0809 04:40:26.110236 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:26.110278 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:26.111493 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:26.111964 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:26.111970 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:26.112325 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:26.112783 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:26.112795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:26.113858 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:26.113862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:26.115083 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:26.115092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:26.115762 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:26.115765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:26.115904 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:26.116555 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:26.116582 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:26.118406 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:26.120492 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:26.120497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:26.120507 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:26.120749 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:26.121091 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:26.121094 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:26.121095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:26.121282 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:26.121809 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:26.121814 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:26.122041 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 04:40:26.122224 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 04:40:26.122539 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
I20260809 04:40:26.124598 10544 heartbeater.cc:347] Connected to a master server at 127.5.222.254:45781
I20260809 04:40:26.124881 10544 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:26.125424 10544 heartbeater.cc:511] Master 127.5.222.254:45781 requested a full tablet report, sending...
I20260809 04:40:26.126488 10174 ts_manager.cc:194] Registered new tserver with Master: 2ff54dd5e1fa482d89cf3b46d79b53e9 (127.5.222.194:38879)
I20260809 04:40:26.127110 10174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:46461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:40:26.142093 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:45781
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:26.252101 10549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:26.252408 10549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:26.252522 10549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:26.256901 10549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:26.256994 10549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:26.257082 10549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:26.262333 10549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:45781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:26.263356 10549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:26.264377 10549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:40:26.274057 10549 server_base.cc:1034] running on GCE node
W20260809 04:40:26.274020 10558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:26.273994 10555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:26.274003 10556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:26.274739 10549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:26.275344 10549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:26.276502 10549 hybrid_clock.cc:648] HybridClock initialized: now 1786250426276474 us; error 41 us; skew 500 ppm
Aug 09 04:40:26 dist-test-slave-vxj2 krb5kdc[10122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250426, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:26.279230 10549 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:40:26.280119 10549 webserver.cc:460] Webserver started at http://127.5.222.195:41903/ using document root <none> and password file <none>
I20260809 04:40:26.280658 10549 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:26.280745 10549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:26.280973 10549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:26.283023 10549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "14a4bb103894462a94bfca32ca55a0dd"
format_stamp: "Formatted at 2026-08-09 04:40:26 on dist-test-slave-vxj2"
server_key: "4a1143fd755d670a6a2001d9ca9b150b"
server_key_iv: "951f3953295a6f9694c003bf8e4079a7"
server_key_version: "encryptionkey@0"
I20260809 04:40:26.283587 10549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "14a4bb103894462a94bfca32ca55a0dd"
format_stamp: "Formatted at 2026-08-09 04:40:26 on dist-test-slave-vxj2"
server_key: "4a1143fd755d670a6a2001d9ca9b150b"
server_key_iv: "951f3953295a6f9694c003bf8e4079a7"
server_key_version: "encryptionkey@0"
I20260809 04:40:26.287343 10549 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:40:26.289774 10566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:26.291043 10549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:26.291178 10549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "14a4bb103894462a94bfca32ca55a0dd"
format_stamp: "Formatted at 2026-08-09 04:40:26 on dist-test-slave-vxj2"
server_key: "4a1143fd755d670a6a2001d9ca9b150b"
server_key_iv: "951f3953295a6f9694c003bf8e4079a7"
server_key_version: "encryptionkey@0"
I20260809 04:40:26.291301 10549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:26.333396 10549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:26.336499 10549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:26.336659 10549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:26.337198 10549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:26.338104 10549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:26.338158 10549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:26.338198 10549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:26.338255 10549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:26.347553 10549 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:40345
I20260809 04:40:26.347579 10696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:40345 every 8 connection(s)
I20260809 04:40:26.348483 10549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
Aug 09 04:40:26 dist-test-slave-vxj2 krb5kdc[10122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250426, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:26.358322 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 10549
I20260809 04:40:26.358474 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:40:26.358780 6011 external_mini_cluster.cc:1428] Setting key 603b69d75f774d20400a2bf3e0b13f21
I20260809 04:40:26.363268 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:26.350455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:46401 (local address 127.5.222.254:45781)
0809 04:40:26.350606 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:26.350610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:26.351453 (+ 843us) server_negotiation.cc:407] Connection header received
0809 04:40:26.352860 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:26.352862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:26.352908 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:26.352984 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:26.354218 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:26.354745 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:26.355987 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:26.356189 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:26.356261 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:26.359066 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:26.359074 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:26.359078 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:26.359109 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:26.360934 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:26.361568 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:26.361572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:26.361574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:26.361666 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:26.362046 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:26.362050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:26.362052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:26.362257 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:26.362377 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:26.362925 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 04:40:26.363057 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:40:26.364049 10701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:26.350800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:45781 (local address 127.5.222.195:46401)
0809 04:40:26.351286 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:26.351332 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:26.352663 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:26.353177 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:26.353184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:26.353583 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:26.354078 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:26.354092 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:26.354885 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:26.354889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:26.355873 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:26.355884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:26.356335 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:26.356338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:26.356493 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:26.357061 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:26.357077 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:26.358878 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:26.361085 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:26.361091 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:26.361105 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:26.361451 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:26.361772 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:26.361776 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:26.361778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:26.361931 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:26.362380 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:26.362390 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:26.362660 (+ 270us) client_negotiation.cc:769] Sending connection context
0809 04:40:26.362861 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 04:40:26.363200 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":127,"threads_started":1}
I20260809 04:40:26.365212 10697 heartbeater.cc:347] Connected to a master server at 127.5.222.254:45781
I20260809 04:40:26.365504 10697 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:26.366032 10697 heartbeater.cc:511] Master 127.5.222.254:45781 requested a full tablet report, sending...
I20260809 04:40:26.367121 10174 ts_manager.cc:194] Registered new tserver with Master: 14a4bb103894462a94bfca32ca55a0dd (127.5.222.195:40345)
I20260809 04:40:26.367756 10174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:46401
I20260809 04:40:26.372556 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:40:26 dist-test-slave-vxj2 krb5kdc[10122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:40:26 dist-test-slave-vxj2 krb5kdc[10122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.193@KRBTEST.COM
I20260809 04:40:26.400017 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:26.390414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46438 (local address 127.5.222.193:38393)
0809 04:40:26.390803 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:26.390818 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:26.390842 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:40:26.390893 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:26.390897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:26.391089 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:26.391209 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:26.391954 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:26.392732 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:26.393590 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:26.393962 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:26.394048 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:26.396492 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:26.396502 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:26.396514 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:26.396537 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:26.398364 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:26.398747 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:26.398753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:26.398754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:26.398842 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:26.399070 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:26.399072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:26.399073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:26.399315 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:26.399465 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:26.399631 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 04:40:26.399748 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":131,"threads_started":1}
Aug 09 04:40:26 dist-test-slave-vxj2 krb5kdc[10122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:26.410681 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:26.403141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.5.222.254:45781)
0809 04:40:26.403293 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:26.403296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:26.403310 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:40:26.403371 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:26.403373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:26.403410 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:26.403480 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:26.404009 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:26.404504 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:26.405296 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:26.405510 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:26.405588 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:26.407891 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:26.407894 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:26.407896 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:26.407921 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:26.409143 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:26.409553 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:26.409555 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:26.409557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:26.409635 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:26.409857 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:26.409859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:26.409860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:26.410065 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:26.410173 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:26.410357 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 04:40:26.410491 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 04:40:26.411173 10174 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49782
I20260809 04:40:26.411830 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 10242
I20260809 04:40:26.419042 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 10394
I20260809 04:40:26.426028 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 10549
I20260809 04:40:26.432807 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 10139
2026-08-09T04:40:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2366 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:26 dist-test-slave-vxj2 krb5kdc[10711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:26 dist-test-slave-vxj2 krb5kdc[10711](info): set up 2 sockets
Aug 09 04:40:26 dist-test-slave-vxj2 krb5kdc[10711](info): commencing operation
krb5kdc: starting...
W20260809 04:40:27.709425 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:27 dist-test-slave-vxj2 krb5kdc[10711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:40:27.910341 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:37091
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:37091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:28.114239 10728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:28.114739 10728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:28.114918 10728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:28.121285 10728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:28.121407 10728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:28.121492 10728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:28.121598 10728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:28.121696 10728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:28.130283 10728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:37091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:37091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:28.132189 10728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:28.133986 10728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:28.147565 10733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:28.147614 10736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:28.147614 10734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:28.148033 10728 server_base.cc:1034] running on GCE node
I20260809 04:40:28.149006 10728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:28.150751 10728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:28.151995 10728 hybrid_clock.cc:648] HybridClock initialized: now 1786250428151971 us; error 37 us; skew 500 ppm
Aug 09 04:40:28 dist-test-slave-vxj2 krb5kdc[10711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250428, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:28.160600 10728 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:40:28.161893 10728 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:28.162017 10728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:28.162317 10728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:28.166075 10728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "12e49c25bc7f457c85a20be52ccb4af8"
format_stamp: "Formatted at 2026-08-09 04:40:28 on dist-test-slave-vxj2"
server_key: "3f682232f2cc334a8cd586c09a532600"
server_key_iv: "e454a54abbe08118ae345b5648133b83"
server_key_version: "encryptionkey@0"
I20260809 04:40:28.166976 10728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "12e49c25bc7f457c85a20be52ccb4af8"
format_stamp: "Formatted at 2026-08-09 04:40:28 on dist-test-slave-vxj2"
server_key: "3f682232f2cc334a8cd586c09a532600"
server_key_iv: "e454a54abbe08118ae345b5648133b83"
server_key_version: "encryptionkey@0"
I20260809 04:40:28.173359 10728 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 04:40:28.177820 10743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:28.180053 10728 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:40:28.180322 10728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "12e49c25bc7f457c85a20be52ccb4af8"
format_stamp: "Formatted at 2026-08-09 04:40:28 on dist-test-slave-vxj2"
server_key: "3f682232f2cc334a8cd586c09a532600"
server_key_iv: "e454a54abbe08118ae345b5648133b83"
server_key_version: "encryptionkey@0"
I20260809 04:40:28.180542 10728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:28.215443 10728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:28.219774 10728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:28.219952 10728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:28.227488 10728 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:37091
I20260809 04:40:28.227773 10806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:37091 every 8 connection(s)
I20260809 04:40:28.228225 10728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:28.231495 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:28.231755 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 10728
I20260809 04:40:28.231918 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:28.232208 6011 external_mini_cluster.cc:1428] Setting key 15420818d8e61960a6ffaceab0790c2a
I20260809 04:40:28.238786 10807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8: Bootstrap starting.
I20260809 04:40:28.241359 10807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8: Neither blocks nor log segments found. Creating new log.
Aug 09 04:40:28 dist-test-slave-vxj2 krb5kdc[10711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:28.242111 10807 log.cc:824] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:28.244568 10807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8: No bootstrap required, opened a new log
I20260809 04:40:28.247843 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.233717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.5.222.254:37091)
0809 04:40:28.234143 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:28.234166 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:28.234210 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:40:28.234981 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:28.235006 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:28.235410 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:28.235772 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:28.236696 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.238222 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.239233 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.239900 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.240093 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.242721 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:28.242725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:28.242739 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:28.242763 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:28.244724 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:28.245138 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:28.245143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:28.245166 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:28.245306 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:28.245581 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:28.245584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:28.245586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:28.246079 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:28.246275 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:28.246494 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 04:40:28.246807 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":115,"threads_started":1}
I20260809 04:40:28.249325 10807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e49c25bc7f457c85a20be52ccb4af8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37091 } }
I20260809 04:40:28.249542 10807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:28.249600 10807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e49c25bc7f457c85a20be52ccb4af8, State: Initialized, Role: FOLLOWER
I20260809 04:40:28.250079 10807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e49c25bc7f457c85a20be52ccb4af8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37091 } }
I20260809 04:40:28.250255 10807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:28.250324 10807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:28.250453 10807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:28.251418 10807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e49c25bc7f457c85a20be52ccb4af8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37091 } }
I20260809 04:40:28.251783 10807 leader_election.cc:302] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12e49c25bc7f457c85a20be52ccb4af8; no voters:
I20260809 04:40:28.252117 10807 leader_election.cc:288] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:28.252183 10813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:28.252379 10813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [term 1 LEADER]: Becoming Leader. State: Replica: 12e49c25bc7f457c85a20be52ccb4af8, State: Running, Role: LEADER
I20260809 04:40:28.252691 10813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e49c25bc7f457c85a20be52ccb4af8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37091 } }
I20260809 04:40:28.253443 10807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:28.254733 10814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12e49c25bc7f457c85a20be52ccb4af8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e49c25bc7f457c85a20be52ccb4af8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37091 } } }
I20260809 04:40:28.254942 10814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:28.254426 10815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12e49c25bc7f457c85a20be52ccb4af8. Latest consensus state: current_term: 1 leader_uuid: "12e49c25bc7f457c85a20be52ccb4af8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e49c25bc7f457c85a20be52ccb4af8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37091 } } }
I20260809 04:40:28.255400 10815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8 [sys.catalog]: This master's current role is: LEADER
W20260809 04:40:28.258212 10833 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:40:28.258278 10833 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:40:28.258359 10826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:28.259068 10826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:28.263882 10826 catalog_manager.cc:1347] Generated new cluster ID: a6aeff1996d342efb2063743bc26c2de
I20260809 04:40:28.263954 10826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:28.276486 10826 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:28.277755 10826 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:28.287537 10826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8: Generated new TSK 0
I20260809 04:40:28.288398 10826 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:40:28.319948 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37091
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:28.430564 10840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:28.430835 10840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:28.430881 10840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:28.434871 10840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:28.434926 10840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:28.435025 10840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:28.440217 10840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:28.441263 10840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:28.442279 10840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:28.451598 10846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:28.451650 10845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:28.451650 10848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:28.452047 10840 server_base.cc:1034] running on GCE node
I20260809 04:40:28.452540 10840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:28.453316 10840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:28.454504 10840 hybrid_clock.cc:648] HybridClock initialized: now 1786250428454476 us; error 29 us; skew 500 ppm
Aug 09 04:40:28 dist-test-slave-vxj2 krb5kdc[10711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250428, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:28.457691 10840 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:40:28.458498 10840 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:28.458601 10840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:28.458861 10840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:28.460992 10840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "1cf81fa0ea0f429aaef44f3c721e2807"
format_stamp: "Formatted at 2026-08-09 04:40:28 on dist-test-slave-vxj2"
server_key: "8dd5f70c65b3581588946f79e4be5840"
server_key_iv: "0119e131f8d75d014d15dd7f62f48dfc"
server_key_version: "encryptionkey@0"
I20260809 04:40:28.461674 10840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "1cf81fa0ea0f429aaef44f3c721e2807"
format_stamp: "Formatted at 2026-08-09 04:40:28 on dist-test-slave-vxj2"
server_key: "8dd5f70c65b3581588946f79e4be5840"
server_key_iv: "0119e131f8d75d014d15dd7f62f48dfc"
server_key_version: "encryptionkey@0"
I20260809 04:40:28.465490 10840 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:40:28.467939 10855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:28.469271 10840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:28.469413 10840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "1cf81fa0ea0f429aaef44f3c721e2807"
format_stamp: "Formatted at 2026-08-09 04:40:28 on dist-test-slave-vxj2"
server_key: "8dd5f70c65b3581588946f79e4be5840"
server_key_iv: "0119e131f8d75d014d15dd7f62f48dfc"
server_key_version: "encryptionkey@0"
I20260809 04:40:28.469532 10840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:28.486897 10840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:28.490654 10840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:28.490897 10840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:28.491554 10840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:28.492470 10840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:28.492547 10840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:28.492624 10840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:28.492683 10840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:28.502740 10840 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:46633
I20260809 04:40:28.502754 10986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:46633 every 8 connection(s)
I20260809 04:40:28.503361 10840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:28.506520 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 10840
I20260809 04:40:28.506675 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:28.506995 6011 external_mini_cluster.cc:1428] Setting key a7ffdd264f99723fa2be4553ce94726a
Aug 09 04:40:28 dist-test-slave-vxj2 krb5kdc[10711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250428, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
I20260809 04:40:28.517236 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.505208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:48369 (local address 127.5.222.254:37091)
0809 04:40:28.505359 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:28.505363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:28.506207 (+ 844us) server_negotiation.cc:407] Connection header received
0809 04:40:28.507669 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:28.507672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:28.507728 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:28.507813 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:28.509085 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.509694 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.510834 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.511082 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.511196 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.513661 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:28.513664 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:28.513666 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:28.513690 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:28.515165 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:28.515667 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:28.515671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:28.515673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:28.515755 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:28.516096 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:28.516101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:28.516103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:28.516281 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:28.516388 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:28.516902 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 04:40:28.517036 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:40:28.518113 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.505613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37091 (local address 127.5.222.193:48369)
0809 04:40:28.506065 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:28.506108 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:28.507438 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:28.507998 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:28.508005 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:28.508409 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:28.508943 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:28.508956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.509825 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.509828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:28.510702 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:28.510711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.511238 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.511242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:28.511419 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.512022 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:28.512037 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:28.513490 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:28.515284 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:28.515290 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:28.515304 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:28.515564 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:28.515847 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:28.515851 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:28.515853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:28.516000 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:28.516387 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:28.516395 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:28.516656 (+ 261us) client_negotiation.cc:769] Sending connection context
0809 04:40:28.516860 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 04:40:28.517209 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:40:28.519381 10987 heartbeater.cc:347] Connected to a master server at 127.5.222.254:37091
I20260809 04:40:28.519645 10987 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:28.520097 10987 heartbeater.cc:511] Master 127.5.222.254:37091 requested a full tablet report, sending...
I20260809 04:40:28.521811 10764 ts_manager.cc:194] Registered new tserver with Master: 1cf81fa0ea0f429aaef44f3c721e2807 (127.5.222.193:46633)
I20260809 04:40:28.523016 10764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:48369
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:40:28.536691 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37091
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:28.647754 10992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:28.647976 10992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:28.648018 10992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:28.651908 10992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:28.651954 10992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:28.652011 10992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:28.657635 10992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:28.658622 10992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:28.659550 10992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:28.668730 10998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:28.668707 10997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:28.668715 11000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:28.668922 10992 server_base.cc:1034] running on GCE node
I20260809 04:40:28.669486 10992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:28.670149 10992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:28.671342 10992 hybrid_clock.cc:648] HybridClock initialized: now 1786250428671306 us; error 53 us; skew 500 ppm
Aug 09 04:40:28 dist-test-slave-vxj2 krb5kdc[10711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250428, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:28.674463 10992 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:40:28.675108 10992 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:28.675169 10992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:28.675335 10992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:28.677345 10992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "41d8451455f440fa932ec8ade979a063"
format_stamp: "Formatted at 2026-08-09 04:40:28 on dist-test-slave-vxj2"
server_key: "f30cc91fcd2dd8399f172c1e37b3ba7b"
server_key_iv: "415db2060eaffa87035f3f5a98bcfc60"
server_key_version: "encryptionkey@0"
I20260809 04:40:28.677846 10992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "41d8451455f440fa932ec8ade979a063"
format_stamp: "Formatted at 2026-08-09 04:40:28 on dist-test-slave-vxj2"
server_key: "f30cc91fcd2dd8399f172c1e37b3ba7b"
server_key_iv: "415db2060eaffa87035f3f5a98bcfc60"
server_key_version: "encryptionkey@0"
I20260809 04:40:28.681516 10992 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:40:28.683758 11006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:28.685354 10992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:40:28.685488 10992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "41d8451455f440fa932ec8ade979a063"
format_stamp: "Formatted at 2026-08-09 04:40:28 on dist-test-slave-vxj2"
server_key: "f30cc91fcd2dd8399f172c1e37b3ba7b"
server_key_iv: "415db2060eaffa87035f3f5a98bcfc60"
server_key_version: "encryptionkey@0"
I20260809 04:40:28.685606 10992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:28.697135 10992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:28.700309 10992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:28.700520 10992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:28.701133 10992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:28.702080 10992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:28.702131 10992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:28.702204 10992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:28.702243 10992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:28.711870 10992 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:44813
I20260809 04:40:28.711892 11139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:44813 every 8 connection(s)
I20260809 04:40:28.712488 10992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:28.713449 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 10992
I20260809 04:40:28.713586 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:28.713956 6011 external_mini_cluster.cc:1428] Setting key d926e335e707f213b53d06341d999051
Aug 09 04:40:28 dist-test-slave-vxj2 krb5kdc[10711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250428, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
I20260809 04:40:28.728843 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.714398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:45963 (local address 127.5.222.254:37091)
0809 04:40:28.714576 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:28.714582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:28.715624 (+ 1042us) server_negotiation.cc:407] Connection header received
0809 04:40:28.717534 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:28.717539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:28.717608 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:28.717724 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:28.718904 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.719488 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.720832 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.721351 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.721480 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.724305 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:28.724313 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:28.724316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:28.724348 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:28.726219 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:28.726890 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:28.726893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:28.726894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:28.726979 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:28.727509 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:28.727512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:28.727522 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:28.727807 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:28.727916 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:28.728529 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 04:40:28.728655 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
I20260809 04:40:28.729657 11143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.714884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37091 (local address 127.5.222.194:45963)
0809 04:40:28.715453 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:28.715511 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:28.717269 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:28.717891 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:28.717897 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:28.718238 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:28.718703 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:28.718716 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.719641 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.719644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:28.720664 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:28.720676 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.721512 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.721515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:28.721663 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.722298 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:28.722314 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:28.724120 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:28.726365 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:28.726370 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:28.726384 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:28.726754 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:28.727100 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:28.727103 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:28.727104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:28.727377 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:28.727930 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:28.727936 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:28.728173 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 04:40:28.728394 (+ 221us) client_negotiation.cc:240] Negotiation successful
0809 04:40:28.728666 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":142,"threads_started":1}
I20260809 04:40:28.731099 11140 heartbeater.cc:347] Connected to a master server at 127.5.222.254:37091
I20260809 04:40:28.731390 11140 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:28.731869 11140 heartbeater.cc:511] Master 127.5.222.254:37091 requested a full tablet report, sending...
I20260809 04:40:28.732985 10764 ts_manager.cc:194] Registered new tserver with Master: 41d8451455f440fa932ec8ade979a063 (127.5.222.194:44813)
I20260809 04:40:28.733634 10764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:45963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:40:28.746241 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37091
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:28.857806 11146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:28.858093 11146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:28.858166 11146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:28.862129 11146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:28.862206 11146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:28.862296 11146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:28.867377 11146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:28.868335 11146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:28.869369 11146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:28.878326 11153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:28.878391 11152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:28.878302 11156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:28.878543 11146 server_base.cc:1034] running on GCE node
I20260809 04:40:28.879055 11146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:28.879699 11146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:28.880878 11146 hybrid_clock.cc:648] HybridClock initialized: now 1786250428880849 us; error 46 us; skew 500 ppm
Aug 09 04:40:28 dist-test-slave-vxj2 krb5kdc[10711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250428, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:28.883634 11146 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:40:28.884289 11146 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:28.884349 11146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:28.884510 11146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:28.886544 11146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "9fccd61ba2d5410ba8f04bfd1f560345"
format_stamp: "Formatted at 2026-08-09 04:40:28 on dist-test-slave-vxj2"
server_key: "d8942eeedb01bc7a23c9f77392fee7a2"
server_key_iv: "cb6093cfe89da8de6a27b8954d3ea72e"
server_key_version: "encryptionkey@0"
I20260809 04:40:28.887040 11146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "9fccd61ba2d5410ba8f04bfd1f560345"
format_stamp: "Formatted at 2026-08-09 04:40:28 on dist-test-slave-vxj2"
server_key: "d8942eeedb01bc7a23c9f77392fee7a2"
server_key_iv: "cb6093cfe89da8de6a27b8954d3ea72e"
server_key_version: "encryptionkey@0"
I20260809 04:40:28.890894 11146 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 04:40:28.893193 11162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:28.894356 11146 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:40:28.894484 11146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "9fccd61ba2d5410ba8f04bfd1f560345"
format_stamp: "Formatted at 2026-08-09 04:40:28 on dist-test-slave-vxj2"
server_key: "d8942eeedb01bc7a23c9f77392fee7a2"
server_key_iv: "cb6093cfe89da8de6a27b8954d3ea72e"
server_key_version: "encryptionkey@0"
I20260809 04:40:28.894584 11146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:28.904498 11146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:28.907294 11146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:28.907492 11146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:28.908043 11146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:28.908845 11146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:28.908900 11146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:28.908942 11146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:28.908958 11146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:28.918087 11146 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:44455
I20260809 04:40:28.918111 11289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:44455 every 8 connection(s)
I20260809 04:40:28.918639 11146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:28.922477 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 11146
I20260809 04:40:28.922607 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:40:28.922936 6011 external_mini_cluster.cc:1428] Setting key f2be04c4f12b965009e3dd59b8d4cd88
Aug 09 04:40:28 dist-test-slave-vxj2 krb5kdc[10711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250428, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:28.932011 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.920327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:52111 (local address 127.5.222.254:37091)
0809 04:40:28.920466 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:28.920469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:28.921334 (+ 865us) server_negotiation.cc:407] Connection header received
0809 04:40:28.922569 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:28.922572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:28.922631 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:28.922723 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:28.924099 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.924712 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.925997 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.926208 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.926301 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.928581 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:28.928583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:28.928585 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:28.928608 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:28.930085 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:28.930522 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:28.930524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:28.930526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:28.930595 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:28.930920 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:28.930924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:28.930925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:28.931164 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:28.931275 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:28.931766 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 04:40:28.931882 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:40:28.932700 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.920672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37091 (local address 127.5.222.195:52111)
0809 04:40:28.921109 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:28.921187 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:28.922366 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:28.922921 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:28.922929 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:28.923370 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:28.923941 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:28.923960 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.924845 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.924848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:28.925883 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:28.925893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.926346 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.926349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:28.926513 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.927018 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:28.927033 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:28.928435 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:28.930178 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:28.930182 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:28.930191 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:28.930433 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:28.930673 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:28.930675 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:28.930677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:28.930797 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:28.931275 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:28.931279 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:28.931506 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 04:40:28.931655 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 04:40:28.931884 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":107,"threads_started":1}
I20260809 04:40:28.933905 11290 heartbeater.cc:347] Connected to a master server at 127.5.222.254:37091
I20260809 04:40:28.934135 11290 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:28.934561 11290 heartbeater.cc:511] Master 127.5.222.254:37091 requested a full tablet report, sending...
I20260809 04:40:28.935843 10764 ts_manager.cc:194] Registered new tserver with Master: 9fccd61ba2d5410ba8f04bfd1f560345 (127.5.222.195:44455)
I20260809 04:40:28.936532 10764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:52111
I20260809 04:40:28.936870 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:40:28.946038 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.938867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.5.222.254:37091)
0809 04:40:28.938993 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:28.938996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:28.939233 (+ 237us) server_negotiation.cc:407] Connection header received
0809 04:40:28.939379 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:28.939382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:28.939429 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:28.939513 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:28.940179 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.940670 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.941544 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.941752 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.941861 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.943101 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:28.943104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:28.943106 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:28.943127 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:28.944565 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:28.944919 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:28.944922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:28.944923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:28.944994 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:28.945276 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:28.945278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:28.945280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:28.945472 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:28.945571 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:28.945835 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 04:40:28.945934 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:40:28.948645 10764 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:28.951571 10764 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:28.966245 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.960008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:46633 (local address 127.0.0.1:56452)
0809 04:40:28.960361 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:28.960412 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:28.960538 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:28.961809 (+ 1271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:28.961815 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:28.961842 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:28.961998 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:28.962004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.963191 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.963197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:28.964567 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:28.964578 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.965670 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.965673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:28.965794 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.965840 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 04:40:28.965969 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 04:40:28.966095 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":75,"threads_started":1}
I20260809 04:40:28.966264 11306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.960147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56452 (local address 127.5.222.193:46633)
0809 04:40:28.961303 (+ 1156us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:28.961317 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:28.961342 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:40:28.961423 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:28.961428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:28.961634 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:28.961789 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:28.962137 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.963063 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.964703 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.965547 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.965670 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.965809 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:28.966011 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 04:40:28.966104 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1014,"thread_start_us":159,"threads_started":1}
I20260809 04:40:28.967124 11308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.960689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:44813 (local address 127.0.0.1:40906)
0809 04:40:28.961472 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:28.961483 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:28.961566 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:28.961998 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:28.962000 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:28.962017 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:28.962123 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:28.962130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.963665 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.963668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:28.964757 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:28.964764 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.966621 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.966625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:28.966761 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.966787 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:40:28.966860 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 04:40:28.966939 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260809 04:40:28.967592 11307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.960236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:44455 (local address 127.0.0.1:50044)
0809 04:40:28.960921 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:28.960934 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:28.961997 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:28.962517 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:28.962521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:28.962540 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:28.962629 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:28.962635 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.964292 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.964295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:28.965368 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:28.965380 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.966258 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.966262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:28.966675 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.966765 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 04:40:28.966843 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 04:40:28.966925 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":47,"thread_start_us":105,"threads_started":1}
I20260809 04:40:28.967604 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.961092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40906 (local address 127.5.222.194:44813)
0809 04:40:28.961460 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:28.961470 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:28.961546 (+ 76us) server_negotiation.cc:407] Connection header received
0809 04:40:28.961659 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:28.961664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:28.961848 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:28.962003 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:28.962214 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.963380 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.964872 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.965547 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.965657 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.965792 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:28.967391 (+ 1599us) server_negotiation.cc:299] Negotiation successful
0809 04:40:28.967488 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":154,"threads_started":1}
I20260809 04:40:28.968430 11309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:28.960741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.5.222.195:44455)
0809 04:40:28.961799 (+ 1058us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:28.961811 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:28.961832 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:40:28.962170 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:28.962176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:28.962364 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:28.962514 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:28.962749 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.964150 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.965505 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:28.966133 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:28.966257 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:28.966377 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:28.967061 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 04:40:28.967152 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":938,"thread_start_us":113,"threads_started":1}
I20260809 04:40:28.969585 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 17c22441720c427a904a09faf50cc2f7 (DEFAULT_TABLE table=test-table [id=5ba8f9b6f1594ac79d830a61e569a77e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:28.970072 11065 tablet_service.cc:1467] Processing CreateTablet for tablet 17c22441720c427a904a09faf50cc2f7 (DEFAULT_TABLE table=test-table [id=5ba8f9b6f1594ac79d830a61e569a77e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:28.970600 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17c22441720c427a904a09faf50cc2f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:28.971073 11065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17c22441720c427a904a09faf50cc2f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:28.976974 11311 tablet_bootstrap.cc:492] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807: Bootstrap starting.
I20260809 04:40:28.976999 11312 tablet_bootstrap.cc:492] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063: Bootstrap starting.
I20260809 04:40:28.979029 11312 tablet_bootstrap.cc:654] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:28.979190 11311 tablet_bootstrap.cc:654] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:28.980113 11312 log.cc:824] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:28.979974 11215 tablet_service.cc:1467] Processing CreateTablet for tablet 17c22441720c427a904a09faf50cc2f7 (DEFAULT_TABLE table=test-table [id=5ba8f9b6f1594ac79d830a61e569a77e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:28.980384 11311 log.cc:824] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:28.981034 11215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17c22441720c427a904a09faf50cc2f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:28.982252 11311 tablet_bootstrap.cc:492] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807: No bootstrap required, opened a new log
I20260809 04:40:28.982474 11311 ts_tablet_manager.cc:1397] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 04:40:28.985296 11311 raft_consensus.cc:348] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 46633 } } peers { permanent_uuid: "41d8451455f440fa932ec8ade979a063" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44813 } } peers { permanent_uuid: "9fccd61ba2d5410ba8f04bfd1f560345" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44455 } }
I20260809 04:40:28.985582 11311 raft_consensus.cc:374] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:28.985687 11311 raft_consensus.cc:729] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cf81fa0ea0f429aaef44f3c721e2807, State: Initialized, Role: FOLLOWER
I20260809 04:40:28.986195 11311 consensus_queue.cc:260] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 46633 } } peers { permanent_uuid: "41d8451455f440fa932ec8ade979a063" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44813 } } peers { permanent_uuid: "9fccd61ba2d5410ba8f04bfd1f560345" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44455 } }
I20260809 04:40:28.986979 10987 heartbeater.cc:503] Master 127.5.222.254:37091 was elected leader, sending a full tablet report...
I20260809 04:40:28.987183 11312 tablet_bootstrap.cc:492] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063: No bootstrap required, opened a new log
I20260809 04:40:28.987236 11311 ts_tablet_manager.cc:1428] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 04:40:28.987368 11312 ts_tablet_manager.cc:1397] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260809 04:40:28.989487 11318 tablet_bootstrap.cc:492] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345: Bootstrap starting.
I20260809 04:40:28.990964 11312 raft_consensus.cc:348] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 46633 } } peers { permanent_uuid: "41d8451455f440fa932ec8ade979a063" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44813 } } peers { permanent_uuid: "9fccd61ba2d5410ba8f04bfd1f560345" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44455 } }
I20260809 04:40:28.991221 11312 raft_consensus.cc:374] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:28.991293 11312 raft_consensus.cc:729] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41d8451455f440fa932ec8ade979a063, State: Initialized, Role: FOLLOWER
I20260809 04:40:28.991729 11318 tablet_bootstrap.cc:654] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:28.991724 11312 consensus_queue.cc:260] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 46633 } } peers { permanent_uuid: "41d8451455f440fa932ec8ade979a063" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44813 } } peers { permanent_uuid: "9fccd61ba2d5410ba8f04bfd1f560345" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44455 } }
I20260809 04:40:28.992409 11140 heartbeater.cc:503] Master 127.5.222.254:37091 was elected leader, sending a full tablet report...
I20260809 04:40:28.992650 11312 ts_tablet_manager.cc:1428] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 04:40:28.992853 11318 log.cc:824] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:28.994871 11318 tablet_bootstrap.cc:492] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345: No bootstrap required, opened a new log
I20260809 04:40:28.995069 11318 ts_tablet_manager.cc:1397] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 04:40:28.998420 11318 raft_consensus.cc:348] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 46633 } } peers { permanent_uuid: "41d8451455f440fa932ec8ade979a063" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44813 } } peers { permanent_uuid: "9fccd61ba2d5410ba8f04bfd1f560345" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44455 } }
I20260809 04:40:28.998636 11318 raft_consensus.cc:374] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:28.998709 11318 raft_consensus.cc:729] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fccd61ba2d5410ba8f04bfd1f560345, State: Initialized, Role: FOLLOWER
I20260809 04:40:28.999110 11318 consensus_queue.cc:260] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 46633 } } peers { permanent_uuid: "41d8451455f440fa932ec8ade979a063" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44813 } } peers { permanent_uuid: "9fccd61ba2d5410ba8f04bfd1f560345" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44455 } }
I20260809 04:40:28.999779 11290 heartbeater.cc:503] Master 127.5.222.254:37091 was elected leader, sending a full tablet report...
I20260809 04:40:29.000088 11318 ts_tablet_manager.cc:1428] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
W20260809 04:40:29.004386 10988 tablet.cc:2369] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:40:29.105809 11317 raft_consensus.cc:482] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:29.106021 11317 raft_consensus.cc:504] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 46633 } } peers { permanent_uuid: "41d8451455f440fa932ec8ade979a063" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44813 } } peers { permanent_uuid: "9fccd61ba2d5410ba8f04bfd1f560345" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44455 } }
I20260809 04:40:29.107131 11317 leader_election.cc:288] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41d8451455f440fa932ec8ade979a063 (127.5.222.194:44813), 9fccd61ba2d5410ba8f04bfd1f560345 (127.5.222.195:44455)
I20260809 04:40:29.112766 11309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:29.107535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:60377 (local address 127.5.222.195:44455)
0809 04:40:29.107664 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:29.107669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:29.107927 (+ 258us) server_negotiation.cc:407] Connection header received
0809 04:40:29.108108 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:29.108113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:29.108170 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:29.108224 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:29.108725 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:29.109498 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:29.111399 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:29.112042 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:29.112154 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:29.112206 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:29.112550 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 04:40:29.112634 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:40:29.112766 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:29.107535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:43857 (local address 127.5.222.194:44813)
0809 04:40:29.107664 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:29.107669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:29.108006 (+ 337us) server_negotiation.cc:407] Connection header received
0809 04:40:29.108115 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:29.108121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:29.108185 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:29.108329 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:29.108669 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:29.109498 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:29.111399 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:29.112021 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:29.112154 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:29.112206 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:29.112550 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 04:40:29.112634 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:40:29.112811 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:29.107408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:44455 (local address 127.5.222.193:60377)
0809 04:40:29.107806 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:29.107824 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:29.107949 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:29.108376 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:29.108379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:29.108436 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:29.108557 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:29.108564 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:29.109633 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:29.109637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:29.111245 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:29.111256 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:29.112163 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:29.112166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:29.112325 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:29.112397 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 04:40:29.112481 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:40:29.112616 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":227,"spinlock_wait_cycles":1920,"thread_start_us":123,"threads_started":1}
I20260809 04:40:29.112810 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:29.107424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:44813 (local address 127.5.222.193:43857)
0809 04:40:29.107921 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:29.107933 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:29.108019 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:29.108352 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:29.108355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:29.108375 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:29.108540 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:29.108545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:29.109633 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:29.109637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:29.111245 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:29.111256 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:29.112163 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:29.112166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:29.112354 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:29.112402 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 04:40:29.112481 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 04:40:29.112614 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":278,"spinlock_wait_cycles":51200,"thread_start_us":97,"threads_started":1}
I20260809 04:40:29.113552 11236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17c22441720c427a904a09faf50cc2f7" candidate_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fccd61ba2d5410ba8f04bfd1f560345" is_pre_election: true
I20260809 04:40:29.113926 11236 raft_consensus.cc:2454] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cf81fa0ea0f429aaef44f3c721e2807 in term 0.
I20260809 04:40:29.114048 11086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17c22441720c427a904a09faf50cc2f7" candidate_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41d8451455f440fa932ec8ade979a063" is_pre_election: true
I20260809 04:40:29.114362 11086 raft_consensus.cc:2454] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cf81fa0ea0f429aaef44f3c721e2807 in term 0.
I20260809 04:40:29.114465 10856 leader_election.cc:302] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cf81fa0ea0f429aaef44f3c721e2807, 9fccd61ba2d5410ba8f04bfd1f560345; no voters:
I20260809 04:40:29.114792 11317 raft_consensus.cc:2781] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:29.114885 11317 raft_consensus.cc:482] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:29.114929 11317 raft_consensus.cc:3037] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:29.115962 11317 raft_consensus.cc:504] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 46633 } } peers { permanent_uuid: "41d8451455f440fa932ec8ade979a063" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44813 } } peers { permanent_uuid: "9fccd61ba2d5410ba8f04bfd1f560345" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44455 } }
I20260809 04:40:29.116304 11317 leader_election.cc:288] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [CANDIDATE]: Term 1 election: Requested vote from peers 41d8451455f440fa932ec8ade979a063 (127.5.222.194:44813), 9fccd61ba2d5410ba8f04bfd1f560345 (127.5.222.195:44455)
I20260809 04:40:29.116730 11086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17c22441720c427a904a09faf50cc2f7" candidate_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41d8451455f440fa932ec8ade979a063"
I20260809 04:40:29.116756 11236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17c22441720c427a904a09faf50cc2f7" candidate_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fccd61ba2d5410ba8f04bfd1f560345"
I20260809 04:40:29.116842 11086 raft_consensus.cc:3037] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:29.116886 11236 raft_consensus.cc:3037] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:29.117956 11086 raft_consensus.cc:2454] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cf81fa0ea0f429aaef44f3c721e2807 in term 1.
I20260809 04:40:29.117961 11236 raft_consensus.cc:2454] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cf81fa0ea0f429aaef44f3c721e2807 in term 1.
I20260809 04:40:29.118257 10859 leader_election.cc:302] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cf81fa0ea0f429aaef44f3c721e2807, 41d8451455f440fa932ec8ade979a063; no voters:
I20260809 04:40:29.118444 11317 raft_consensus.cc:2781] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:29.118669 11317 raft_consensus.cc:686] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 1 LEADER]: Becoming Leader. State: Replica: 1cf81fa0ea0f429aaef44f3c721e2807, State: Running, Role: LEADER
I20260809 04:40:29.119000 11317 consensus_queue.cc:237] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 46633 } } peers { permanent_uuid: "41d8451455f440fa932ec8ade979a063" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44813 } } peers { permanent_uuid: "9fccd61ba2d5410ba8f04bfd1f560345" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44455 } }
I20260809 04:40:29.122141 10763 catalog_manager.cc:5495] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cf81fa0ea0f429aaef44f3c721e2807 (127.5.222.193). New cstate: current_term: 1 leader_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cf81fa0ea0f429aaef44f3c721e2807" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 46633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41d8451455f440fa932ec8ade979a063" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fccd61ba2d5410ba8f04bfd1f560345" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44455 } health_report { overall_health: UNKNOWN } } }
I20260809 04:40:29.153883 11306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:29.150187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56464 (local address 127.5.222.193:46633)
0809 04:40:29.150349 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:29.150354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:29.150368 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:40:29.150543 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:29.150546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:29.150600 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:29.150713 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:29.150986 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:29.151483 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:29.152391 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:29.152613 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:29.152726 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:29.153047 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:29.153464 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:29.153605 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:29.153667 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 04:40:29.153733 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:40:29.159932 11086 raft_consensus.cc:1261] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063 [term 1 FOLLOWER]: Refusing update from remote peer 1cf81fa0ea0f429aaef44f3c721e2807: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:29.159932 11236 raft_consensus.cc:1261] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345 [term 1 FOLLOWER]: Refusing update from remote peer 1cf81fa0ea0f429aaef44f3c721e2807: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:29.160633 11325 consensus_queue.cc:1035] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fccd61ba2d5410ba8f04bfd1f560345" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:29.160787 11317 consensus_queue.cc:1035] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41d8451455f440fa932ec8ade979a063" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:29.176335 10763 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:29.176524 10763 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:29.178949 10763 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 12e49c25bc7f457c85a20be52ccb4af8: Sending DeleteTablet for 3 replicas of tablet 17c22441720c427a904a09faf50cc2f7
I20260809 04:40:29.179745 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet 17c22441720c427a904a09faf50cc2f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:29 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:56452
I20260809 04:40:29.179803 11215 tablet_service.cc:1514] Processing DeleteTablet for tablet 17c22441720c427a904a09faf50cc2f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:29 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:50044
I20260809 04:40:29.180229 11341 tablet_replica.cc:330] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807: stopping tablet replica
I20260809 04:40:29.180289 11342 tablet_replica.cc:330] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345: stopping tablet replica
I20260809 04:40:29.180394 11065 tablet_service.cc:1514] Processing DeleteTablet for tablet 17c22441720c427a904a09faf50cc2f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:29 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:40906
I20260809 04:40:29.180601 11342 raft_consensus.cc:2229] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:29.180745 11341 raft_consensus.cc:2229] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:29.180913 11342 raft_consensus.cc:2258] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:29.181317 11341 raft_consensus.cc:2258] T 17c22441720c427a904a09faf50cc2f7 P 1cf81fa0ea0f429aaef44f3c721e2807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:29.181345 11343 tablet_replica.cc:330] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063: stopping tablet replica
I20260809 04:40:29.181586 11343 raft_consensus.cc:2229] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:29.181737 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 10840
I20260809 04:40:29.186326 11343 raft_consensus.cc:2258] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:29.186869 11342 ts_tablet_manager.cc:1905] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:29.189109 11343 ts_tablet_manager.cc:1905] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:29.189868 11342 ts_tablet_manager.cc:1918] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:29.189998 11342 log.cc:1196] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/17c22441720c427a904a09faf50cc2f7
I20260809 04:40:29.190390 11342 ts_tablet_manager.cc:1939] T 17c22441720c427a904a09faf50cc2f7 P 9fccd61ba2d5410ba8f04bfd1f560345: Deleting consensus metadata
I20260809 04:40:29.191732 10744 catalog_manager.cc:4841] TS 9fccd61ba2d5410ba8f04bfd1f560345 (127.5.222.195:44455): tablet 17c22441720c427a904a09faf50cc2f7 (table test-table [id=5ba8f9b6f1594ac79d830a61e569a77e]) successfully deleted
I20260809 04:40:29.192675 11343 ts_tablet_manager.cc:1918] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:29.192787 11343 log.cc:1196] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/17c22441720c427a904a09faf50cc2f7
I20260809 04:40:29.193120 11343 ts_tablet_manager.cc:1939] T 17c22441720c427a904a09faf50cc2f7 P 41d8451455f440fa932ec8ade979a063: Deleting consensus metadata
W20260809 04:40:29.195489 10748 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:46633 (error 108)
I20260809 04:40:29.196316 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 10992
I20260809 04:40:29.196532 10748 catalog_manager.cc:4841] TS 41d8451455f440fa932ec8ade979a063 (127.5.222.194:44813): tablet 17c22441720c427a904a09faf50cc2f7 (table test-table [id=5ba8f9b6f1594ac79d830a61e569a77e]) successfully deleted
I20260809 04:40:29.198050 11307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:29.196364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:46633 (local address 127.0.0.1:56470)
0809 04:40:29.197825 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:29.197954 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1361}
W20260809 04:40:29.198629 10748 catalog_manager.cc:4584] TS 1cf81fa0ea0f429aaef44f3c721e2807 (127.5.222.193:46633): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17c22441720c427a904a09faf50cc2f7: Network error: Client connection negotiation failed: client connection to 127.5.222.193:46633: connect: Connection refused (error 111)
I20260809 04:40:29.207631 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 11146
I20260809 04:40:29.214627 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 10728
2026-08-09T04:40:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2781 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T04:40:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:29Z Disabled control of system clock
I20260809 04:40:29.248165 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:37131
--webserver_interface=127.5.222.254
--webserver_port=0
--builtin_ntp_servers=127.5.222.212:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:37131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 04:40:29.365422 11348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:29.365670 11348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:29.365710 11348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:29.369803 11348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:29.369863 11348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:29.369879 11348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:29.369895 11348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:29.369911 11348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:29.374915 11348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:37131
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:37131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:29.376101 11348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:29.377547 11348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:29.388644 11357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:29.388682 11353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:29.388644 11354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:29.389878 11348 server_base.cc:1034] running on GCE node
I20260809 04:40:29.390456 11348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:29.391829 11348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:29.393035 11348 hybrid_clock.cc:648] HybridClock initialized: now 1786250429393006 us; error 48 us; skew 500 ppm
I20260809 04:40:29.394922 11348 webserver.cc:460] Webserver started at http://127.5.222.254:39291/ using document root <none> and password file <none>
I20260809 04:40:29.395514 11348 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:29.395578 11348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:29.395805 11348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:29.397971 11348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "303df4f3990a46018cc2d327fc4b928a"
format_stamp: "Formatted at 2026-08-09 04:40:29 on dist-test-slave-vxj2"
server_key: "842db54e1a4093f2418057e1744806dc"
server_key_iv: "31e920be5ad7a56622908ce85f3cc49f"
server_key_version: "encryptionkey@0"
I20260809 04:40:29.398556 11348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "303df4f3990a46018cc2d327fc4b928a"
format_stamp: "Formatted at 2026-08-09 04:40:29 on dist-test-slave-vxj2"
server_key: "842db54e1a4093f2418057e1744806dc"
server_key_iv: "31e920be5ad7a56622908ce85f3cc49f"
server_key_version: "encryptionkey@0"
I20260809 04:40:29.402292 11348 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 04:40:29.404754 11363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:29.406045 11348 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:40:29.406199 11348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "303df4f3990a46018cc2d327fc4b928a"
format_stamp: "Formatted at 2026-08-09 04:40:29 on dist-test-slave-vxj2"
server_key: "842db54e1a4093f2418057e1744806dc"
server_key_iv: "31e920be5ad7a56622908ce85f3cc49f"
server_key_version: "encryptionkey@0"
I20260809 04:40:29.406308 11348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:29.411090 11348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:29.411710 11348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:29.411896 11348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:29.420087 11348 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:37131
I20260809 04:40:29.420117 11428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:37131 every 8 connection(s)
I20260809 04:40:29.421079 11348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:29.424049 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 11348
I20260809 04:40:29.424190 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:29.424593 6011 external_mini_cluster.cc:1428] Setting key ae079f64306ab9d86baa7dcb5e622cf6
I20260809 04:40:29.424762 11430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:29.430540 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:29.426118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.5.222.254:37131)
0809 04:40:29.426820 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:29.426837 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:29.426874 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:40:29.427629 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:29.427647 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:29.428044 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:29.428328 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:29.429256 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:29.429261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:29.429271 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:29.429282 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:29.429337 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:29.429352 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:29.429477 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:29.429572 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 04:40:29.429832 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":146,"threads_started":1}
I20260809 04:40:29.432174 11430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a: Bootstrap starting.
I20260809 04:40:29.434475 11430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:29.435190 11430 log.cc:824] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:29.437300 11430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a: No bootstrap required, opened a new log
I20260809 04:40:29.440867 11430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303df4f3990a46018cc2d327fc4b928a" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37131 } }
I20260809 04:40:29.441093 11430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:29.441190 11430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 303df4f3990a46018cc2d327fc4b928a, State: Initialized, Role: FOLLOWER
I20260809 04:40:29.441684 11430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303df4f3990a46018cc2d327fc4b928a" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37131 } }
I20260809 04:40:29.441805 11430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:29.441891 11430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:29.441959 11430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:29.442853 11430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303df4f3990a46018cc2d327fc4b928a" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37131 } }
I20260809 04:40:29.443226 11430 leader_election.cc:302] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 303df4f3990a46018cc2d327fc4b928a; no voters:
I20260809 04:40:29.443579 11430 leader_election.cc:288] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:29.443660 11438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:29.443837 11438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [term 1 LEADER]: Becoming Leader. State: Replica: 303df4f3990a46018cc2d327fc4b928a, State: Running, Role: LEADER
I20260809 04:40:29.444126 11438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303df4f3990a46018cc2d327fc4b928a" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37131 } }
I20260809 04:40:29.444872 11430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:29.445919 11439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "303df4f3990a46018cc2d327fc4b928a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303df4f3990a46018cc2d327fc4b928a" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37131 } } }
I20260809 04:40:29.445935 11440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 303df4f3990a46018cc2d327fc4b928a. Latest consensus state: current_term: 1 leader_uuid: "303df4f3990a46018cc2d327fc4b928a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303df4f3990a46018cc2d327fc4b928a" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37131 } } }
I20260809 04:40:29.446177 11439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:29.446209 11440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:29.447340 11448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:29.450064 11448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:29.455451 11448 catalog_manager.cc:1347] Generated new cluster ID: aa250871f2db4df9af95a03bb0a29d55
I20260809 04:40:29.455513 11448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:29.484617 11448 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:29.485042 11448 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:29.488574 11448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a: Generated new TSK 0
I20260809 04:40:29.489351 11448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:40:29.494397 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37131
--builtin_ntp_servers=127.5.222.212:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 04:40:29.606755 11459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:29.607035 11459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:29.607105 11459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:29.611390 11459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:29.611446 11459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:29.611534 11459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:29.616287 11459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:29.617398 11459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:29.618767 11459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:29.628283 11464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:29.628360 11469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:29.628306 11465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:29.628568 11459 server_base.cc:1034] running on GCE node
I20260809 04:40:29.629029 11459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:29.629685 11459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:29.630873 11459 hybrid_clock.cc:648] HybridClock initialized: now 1786250429630857 us; error 33 us; skew 500 ppm
I20260809 04:40:29.632558 11459 webserver.cc:460] Webserver started at http://127.5.222.193:40939/ using document root <none> and password file <none>
I20260809 04:40:29.633204 11459 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:29.633298 11459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:29.633512 11459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:29.635479 11459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "868b04d2561040e9bd4f4fa78c7a569f"
format_stamp: "Formatted at 2026-08-09 04:40:29 on dist-test-slave-vxj2"
server_key: "e45be6be56b27ec5d1148dee82da4312"
server_key_iv: "984c363234ee718685bd4c52e4ff8800"
server_key_version: "encryptionkey@0"
I20260809 04:40:29.636046 11459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "868b04d2561040e9bd4f4fa78c7a569f"
format_stamp: "Formatted at 2026-08-09 04:40:29 on dist-test-slave-vxj2"
server_key: "e45be6be56b27ec5d1148dee82da4312"
server_key_iv: "984c363234ee718685bd4c52e4ff8800"
server_key_version: "encryptionkey@0"
I20260809 04:40:29.639572 11459 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:40:29.641987 11477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:29.643165 11459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 04:40:29.643297 11459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "868b04d2561040e9bd4f4fa78c7a569f"
format_stamp: "Formatted at 2026-08-09 04:40:29 on dist-test-slave-vxj2"
server_key: "e45be6be56b27ec5d1148dee82da4312"
server_key_iv: "984c363234ee718685bd4c52e4ff8800"
server_key_version: "encryptionkey@0"
I20260809 04:40:29.643409 11459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:29.648480 11459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:29.649056 11459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:29.649256 11459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:29.649806 11459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:29.650727 11459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:29.650807 11459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:29.650877 11459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:29.650928 11459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:29.659854 11459 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:45163
I20260809 04:40:29.659876 11610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:45163 every 8 connection(s)
I20260809 04:40:29.660743 11459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:29.666827 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:29.662594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:58477 (local address 127.5.222.254:37131)
0809 04:40:29.662735 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:29.662739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:29.663881 (+ 1142us) server_negotiation.cc:407] Connection header received
0809 04:40:29.665301 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:29.665304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:29.665339 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:29.665415 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:29.666399 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:29.666402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:29.666404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:29.666417 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:29.666448 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:29.666456 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:29.666530 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:29.666652 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 04:40:29.666688 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:40:29.667624 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:29.662976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37131 (local address 127.5.222.193:58477)
0809 04:40:29.663711 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:29.663765 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:29.665056 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:29.665564 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:29.665570 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:29.665651 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:29.666149 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:40:29.666151 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:29.666189 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:40:29.666251 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:29.666533 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:29.666539 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:29.666543 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:40:29.666668 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 04:40:29.666846 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":121,"threads_started":1}
I20260809 04:40:29.668799 11611 heartbeater.cc:347] Connected to a master server at 127.5.222.254:37131
I20260809 04:40:29.669036 11611 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:29.669446 11611 heartbeater.cc:511] Master 127.5.222.254:37131 requested a full tablet report, sending...
I20260809 04:40:29.670953 11384 ts_manager.cc:194] Registered new tserver with Master: 868b04d2561040e9bd4f4fa78c7a569f (127.5.222.193:45163)
I20260809 04:40:29.671067 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 11459
I20260809 04:40:29.671182 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:29.671504 6011 external_mini_cluster.cc:1428] Setting key ce71cc947c9854effb3ea7c4a8f06938
I20260809 04:40:29.673332 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37131
--builtin_ntp_servers=127.5.222.212:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 04:40:29.792418 11614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:29.792717 11614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:29.792790 11614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:29.797019 11614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:29.797094 11614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:29.797230 11614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:29.802239 11614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:29.803421 11614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:29.804903 11614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:29.814958 11623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:29.815050 11621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:29.814960 11619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:29.815316 11614 server_base.cc:1034] running on GCE node
I20260809 04:40:29.815718 11614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:29.816422 11614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:29.817590 11614 hybrid_clock.cc:648] HybridClock initialized: now 1786250429817569 us; error 32 us; skew 500 ppm
I20260809 04:40:29.819681 11614 webserver.cc:460] Webserver started at http://127.5.222.194:37681/ using document root <none> and password file <none>
I20260809 04:40:29.820305 11614 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:29.820372 11614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:29.820587 11614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:29.822757 11614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "167ff7950c5248d9ab7f8d2694e880b8"
format_stamp: "Formatted at 2026-08-09 04:40:29 on dist-test-slave-vxj2"
server_key: "9c5ac71dacda58e5f020878b6c8c5a01"
server_key_iv: "ae15e5d763e56d33b22f6da60452cd27"
server_key_version: "encryptionkey@0"
I20260809 04:40:29.823328 11614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "167ff7950c5248d9ab7f8d2694e880b8"
format_stamp: "Formatted at 2026-08-09 04:40:29 on dist-test-slave-vxj2"
server_key: "9c5ac71dacda58e5f020878b6c8c5a01"
server_key_iv: "ae15e5d763e56d33b22f6da60452cd27"
server_key_version: "encryptionkey@0"
I20260809 04:40:29.827190 11614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:40:29.829667 11632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:29.830855 11614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:29.830993 11614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "167ff7950c5248d9ab7f8d2694e880b8"
format_stamp: "Formatted at 2026-08-09 04:40:29 on dist-test-slave-vxj2"
server_key: "9c5ac71dacda58e5f020878b6c8c5a01"
server_key_iv: "ae15e5d763e56d33b22f6da60452cd27"
server_key_version: "encryptionkey@0"
I20260809 04:40:29.831108 11614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:29.835807 11614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:29.836398 11614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:29.836612 11614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:29.837189 11614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:29.838289 11614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:29.838346 11614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:29.838413 11614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:29.838449 11614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:29.848254 11614 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:41793
I20260809 04:40:29.848341 11759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:41793 every 8 connection(s)
I20260809 04:40:29.849224 11614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:29.849862 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 11614
I20260809 04:40:29.849992 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:29.850270 6011 external_mini_cluster.cc:1428] Setting key b670ed3786f072cfda0aada146a6702b
I20260809 04:40:29.852304 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37131
--builtin_ntp_servers=127.5.222.212:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 04:40:29.859714 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:29.854895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:39447 (local address 127.5.222.254:37131)
0809 04:40:29.855071 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:29.855076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:29.856273 (+ 1197us) server_negotiation.cc:407] Connection header received
0809 04:40:29.857833 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:29.857838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:29.857891 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:29.858010 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:29.859160 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:29.859166 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:29.859171 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:29.859189 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:29.859218 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:29.859232 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:29.859342 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:29.859467 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 04:40:29.859523 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 04:40:29.860482 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:29.855361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37131 (local address 127.5.222.194:39447)
0809 04:40:29.856080 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:29.856133 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:29.857619 (+ 1486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:29.858155 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:29.858161 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:29.858263 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:29.858891 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:40:29.858892 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:29.858930 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:40:29.858995 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:29.859329 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:29.859335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:29.859339 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:40:29.859465 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 04:40:29.859650 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":147,"threads_started":1}
I20260809 04:40:29.861790 11760 heartbeater.cc:347] Connected to a master server at 127.5.222.254:37131
I20260809 04:40:29.862036 11760 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:29.862448 11760 heartbeater.cc:511] Master 127.5.222.254:37131 requested a full tablet report, sending...
I20260809 04:40:29.863528 11384 ts_manager.cc:194] Registered new tserver with Master: 167ff7950c5248d9ab7f8d2694e880b8 (127.5.222.194:41793)
W20260809 04:40:29.970139 11762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:29.970417 11762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:29.970456 11762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:29.974484 11762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:29.974542 11762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:29.974596 11762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:29.979393 11762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:29.980482 11762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:29.981910 11762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:29.992472 11769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:29.992472 11768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:29.992488 11771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:29.992848 11762 server_base.cc:1034] running on GCE node
I20260809 04:40:29.993324 11762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:29.994006 11762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:29.995226 11762 hybrid_clock.cc:648] HybridClock initialized: now 1786250429995212 us; error 35 us; skew 500 ppm
I20260809 04:40:29.997287 11762 webserver.cc:460] Webserver started at http://127.5.222.195:34817/ using document root <none> and password file <none>
I20260809 04:40:29.997880 11762 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:29.997978 11762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:29.998198 11762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:30.000332 11762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "5c5635b8592e4950b6f032192d355f94"
format_stamp: "Formatted at 2026-08-09 04:40:29 on dist-test-slave-vxj2"
server_key: "84d979f4aba8a23dc982995aebd5fa7f"
server_key_iv: "72108e06638b59fa49ce6cf6cd172020"
server_key_version: "encryptionkey@0"
I20260809 04:40:30.000921 11762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "5c5635b8592e4950b6f032192d355f94"
format_stamp: "Formatted at 2026-08-09 04:40:29 on dist-test-slave-vxj2"
server_key: "84d979f4aba8a23dc982995aebd5fa7f"
server_key_iv: "72108e06638b59fa49ce6cf6cd172020"
server_key_version: "encryptionkey@0"
I20260809 04:40:30.004889 11762 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260809 04:40:30.007467 11778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:30.008725 11762 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:30.008879 11762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "5c5635b8592e4950b6f032192d355f94"
format_stamp: "Formatted at 2026-08-09 04:40:29 on dist-test-slave-vxj2"
server_key: "84d979f4aba8a23dc982995aebd5fa7f"
server_key_iv: "72108e06638b59fa49ce6cf6cd172020"
server_key_version: "encryptionkey@0"
I20260809 04:40:30.009011 11762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:30.014217 11762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:30.014850 11762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:30.015038 11762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:30.015614 11762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:30.016547 11762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:30.016618 11762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:30.016687 11762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:30.016741 11762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:30.025913 11762 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:37035
I20260809 04:40:30.025933 11903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:37035 every 8 connection(s)
I20260809 04:40:30.026850 11762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:30.029618 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 11762
I20260809 04:40:30.029743 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:40:30.030097 6011 external_mini_cluster.cc:1428] Setting key aef353de81828817e3a8b370c1ffd055
I20260809 04:40:30.033439 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.028642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:42477 (local address 127.5.222.254:37131)
0809 04:40:30.028787 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:30.028791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:30.030018 (+ 1227us) server_negotiation.cc:407] Connection header received
0809 04:40:30.031605 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:30.031609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:30.031659 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:30.031725 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:30.032931 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:30.032934 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:30.032936 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:30.032947 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:30.032965 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:30.032973 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:30.033086 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:30.033211 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 04:40:30.033249 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:40:30.034282 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.029041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37131 (local address 127.5.222.195:42477)
0809 04:40:30.029837 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:30.029896 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:30.031418 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:30.031895 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:30.031905 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:30.032017 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:30.032691 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:40:30.032693 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:30.032733 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:40:30.032797 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:30.033062 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:30.033066 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:30.033070 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:40:30.033247 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 04:40:30.033442 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":133,"threads_started":1}
I20260809 04:40:30.035378 11904 heartbeater.cc:347] Connected to a master server at 127.5.222.254:37131
I20260809 04:40:30.035562 11904 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:30.035939 11904 heartbeater.cc:511] Master 127.5.222.254:37131 requested a full tablet report, sending...
I20260809 04:40:30.036844 11384 ts_manager.cc:194] Registered new tserver with Master: 5c5635b8592e4950b6f032192d355f94 (127.5.222.195:37035)
I20260809 04:40:30.044509 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:40:30.048396 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.046676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35658 (local address 127.5.222.254:37131)
0809 04:40:30.046801 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:30.046805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:30.047027 (+ 222us) server_negotiation.cc:407] Connection header received
0809 04:40:30.047093 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:30.047095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:30.047139 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:30.047256 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:30.048028 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:30.048031 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:30.048033 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:30.048043 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:30.048062 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:30.048069 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:30.048140 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:30.048218 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 04:40:30.048249 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:40:30.051400 11384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:30.054535 11384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:30.067456 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.063388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:41793 (local address 127.0.0.1:51426)
0809 04:40:30.064363 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:30.064415 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:30.064518 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:30.065766 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:30.065772 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:30.065852 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:30.066613 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:40:30.066615 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:30.066636 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:40:30.066706 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:30.067121 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:30.067124 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:30.067128 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:40:30.067206 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 04:40:30.067281 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":95,"threads_started":1}
I20260809 04:40:30.067482 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.063996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51938 (local address 127.5.222.195:37035)
0809 04:40:30.064422 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:30.064437 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:30.064509 (+ 72us) server_negotiation.cc:407] Connection header received
0809 04:40:30.064686 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:30.064693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:30.064920 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:30.065034 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:30.066904 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:30.066908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:30.066916 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:30.066929 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:30.066965 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:30.066997 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:30.067076 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:30.067263 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 04:40:30.067327 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":118,"threads_started":1}
I20260809 04:40:30.067515 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.064302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40602 (local address 127.5.222.193:45163)
0809 04:40:30.064599 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:30.064610 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:30.064844 (+ 234us) server_negotiation.cc:407] Connection header received
0809 04:40:30.065052 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:30.065060 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:30.065287 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:30.065376 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:30.066850 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:30.066856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:30.066870 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:30.066882 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:30.066935 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:30.066962 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:30.067081 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:30.067231 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 04:40:30.067314 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":110,"threads_started":1}
I20260809 04:40:30.067894 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.063508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.5.222.194:41793)
0809 04:40:30.064694 (+ 1186us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:30.064707 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:30.064727 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:40:30.064772 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:30.064776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:30.064919 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:30.065043 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:30.066862 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:30.066870 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:30.066880 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:30.066893 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:30.066939 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:30.066973 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:30.067076 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:30.067653 (+ 577us) server_negotiation.cc:299] Negotiation successful
0809 04:40:30.067717 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":110,"threads_started":1}
I20260809 04:40:30.068048 11916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.063464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:37035 (local address 127.0.0.1:51938)
0809 04:40:30.064388 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:30.064415 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:30.064561 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:30.065768 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:30.065773 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:30.065852 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:30.066613 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:40:30.066615 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:30.066636 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:40:30.066706 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:30.067144 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:30.067147 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:30.067153 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:40:30.067881 (+ 728us) client_negotiation.cc:240] Negotiation successful
0809 04:40:30.067921 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"thread_start_us":104,"threads_started":1}
I20260809 04:40:30.068303 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.064126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:45163 (local address 127.0.0.1:40602)
0809 04:40:30.064742 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:30.064756 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:30.064907 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:30.065877 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:30.065879 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:30.065892 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:30.066670 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:40:30.066672 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:30.066682 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:40:30.066706 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:30.067053 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:30.067063 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:30.067068 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:40:30.068120 (+ 1052us) client_negotiation.cc:240] Negotiation successful
0809 04:40:30.068157 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":22,"thread_start_us":83,"threads_started":1}
I20260809 04:40:30.071010 11534 tablet_service.cc:1467] Processing CreateTablet for tablet 1362a0c8e1d342128f31e0f311b4e21d (DEFAULT_TABLE table=test-table [id=cc176edd8d0b42de9c6f0eabb28202af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:30.071009 11834 tablet_service.cc:1467] Processing CreateTablet for tablet 1362a0c8e1d342128f31e0f311b4e21d (DEFAULT_TABLE table=test-table [id=cc176edd8d0b42de9c6f0eabb28202af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:30.071009 11685 tablet_service.cc:1467] Processing CreateTablet for tablet 1362a0c8e1d342128f31e0f311b4e21d (DEFAULT_TABLE table=test-table [id=cc176edd8d0b42de9c6f0eabb28202af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:30.072023 11534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1362a0c8e1d342128f31e0f311b4e21d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:30.072041 11834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1362a0c8e1d342128f31e0f311b4e21d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:30.072041 11685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1362a0c8e1d342128f31e0f311b4e21d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:30.078166 11923 tablet_bootstrap.cc:492] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f: Bootstrap starting.
I20260809 04:40:30.079665 11925 tablet_bootstrap.cc:492] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8: Bootstrap starting.
I20260809 04:40:30.081029 11923 tablet_bootstrap.cc:654] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:30.081459 11925 tablet_bootstrap.cc:654] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:30.081501 11926 tablet_bootstrap.cc:492] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94: Bootstrap starting.
I20260809 04:40:30.081986 11923 log.cc:824] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:30.082207 11925 log.cc:824] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:30.083498 11926 tablet_bootstrap.cc:654] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:30.084136 11926 log.cc:824] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:30.084158 11923 tablet_bootstrap.cc:492] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f: No bootstrap required, opened a new log
I20260809 04:40:30.084158 11925 tablet_bootstrap.cc:492] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8: No bootstrap required, opened a new log
I20260809 04:40:30.084360 11923 ts_tablet_manager.cc:1397] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:40:30.084360 11925 ts_tablet_manager.cc:1397] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:40:30.086064 11926 tablet_bootstrap.cc:492] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94: No bootstrap required, opened a new log
I20260809 04:40:30.086258 11926 ts_tablet_manager.cc:1397] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:40:30.089133 11925 raft_consensus.cc:348] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167ff7950c5248d9ab7f8d2694e880b8" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 41793 } } peers { permanent_uuid: "5c5635b8592e4950b6f032192d355f94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 37035 } } peers { permanent_uuid: "868b04d2561040e9bd4f4fa78c7a569f" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 45163 } }
I20260809 04:40:30.089133 11923 raft_consensus.cc:348] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167ff7950c5248d9ab7f8d2694e880b8" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 41793 } } peers { permanent_uuid: "5c5635b8592e4950b6f032192d355f94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 37035 } } peers { permanent_uuid: "868b04d2561040e9bd4f4fa78c7a569f" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 45163 } }
I20260809 04:40:30.089455 11925 raft_consensus.cc:374] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:30.089455 11923 raft_consensus.cc:374] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:30.089524 11923 raft_consensus.cc:729] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 868b04d2561040e9bd4f4fa78c7a569f, State: Initialized, Role: FOLLOWER
I20260809 04:40:30.089524 11925 raft_consensus.cc:729] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 167ff7950c5248d9ab7f8d2694e880b8, State: Initialized, Role: FOLLOWER
I20260809 04:40:30.089401 11926 raft_consensus.cc:348] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167ff7950c5248d9ab7f8d2694e880b8" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 41793 } } peers { permanent_uuid: "5c5635b8592e4950b6f032192d355f94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 37035 } } peers { permanent_uuid: "868b04d2561040e9bd4f4fa78c7a569f" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 45163 } }
I20260809 04:40:30.089664 11926 raft_consensus.cc:374] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:30.089720 11926 raft_consensus.cc:729] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5635b8592e4950b6f032192d355f94, State: Initialized, Role: FOLLOWER
I20260809 04:40:30.090036 11923 consensus_queue.cc:260] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167ff7950c5248d9ab7f8d2694e880b8" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 41793 } } peers { permanent_uuid: "5c5635b8592e4950b6f032192d355f94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 37035 } } peers { permanent_uuid: "868b04d2561040e9bd4f4fa78c7a569f" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 45163 } }
I20260809 04:40:30.090036 11925 consensus_queue.cc:260] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167ff7950c5248d9ab7f8d2694e880b8" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 41793 } } peers { permanent_uuid: "5c5635b8592e4950b6f032192d355f94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 37035 } } peers { permanent_uuid: "868b04d2561040e9bd4f4fa78c7a569f" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 45163 } }
I20260809 04:40:30.090142 11926 consensus_queue.cc:260] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167ff7950c5248d9ab7f8d2694e880b8" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 41793 } } peers { permanent_uuid: "5c5635b8592e4950b6f032192d355f94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 37035 } } peers { permanent_uuid: "868b04d2561040e9bd4f4fa78c7a569f" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 45163 } }
I20260809 04:40:30.090735 11760 heartbeater.cc:503] Master 127.5.222.254:37131 was elected leader, sending a full tablet report...
I20260809 04:40:30.091027 11611 heartbeater.cc:503] Master 127.5.222.254:37131 was elected leader, sending a full tablet report...
I20260809 04:40:30.091099 11923 ts_tablet_manager.cc:1428] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260809 04:40:30.091320 11904 heartbeater.cc:503] Master 127.5.222.254:37131 was elected leader, sending a full tablet report...
I20260809 04:40:30.091717 11925 ts_tablet_manager.cc:1428] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260809 04:40:30.092753 11926 ts_tablet_manager.cc:1428] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260809 04:40:30.100207 11761 tablet.cc:2369] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:40:30.161935 11612 tablet.cc:2369] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:40:30.196792 11932 raft_consensus.cc:482] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:30.197038 11932 raft_consensus.cc:504] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167ff7950c5248d9ab7f8d2694e880b8" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 41793 } } peers { permanent_uuid: "5c5635b8592e4950b6f032192d355f94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 37035 } } peers { permanent_uuid: "868b04d2561040e9bd4f4fa78c7a569f" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 45163 } }
I20260809 04:40:30.198212 11932 leader_election.cc:288] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 167ff7950c5248d9ab7f8d2694e880b8 (127.5.222.194:41793), 868b04d2561040e9bd4f4fa78c7a569f (127.5.222.193:45163)
I20260809 04:40:30.200580 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.198581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:41793 (local address 127.5.222.195:48963)
0809 04:40:30.198738 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:30.198756 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:30.198892 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:30.199381 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:30.199385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:30.199397 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:30.199960 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:40:30.199964 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:30.199979 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:40:30.200014 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:30.200303 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:30.200307 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:30.200310 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 04:40:30.200396 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:40:30.200438 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 04:40:30.200598 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.198698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:48963 (local address 127.5.222.194:41793)
0809 04:40:30.198859 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:30.198865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:30.198883 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:40:30.199194 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:30.199197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:30.199261 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:30.199334 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:30.200140 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:30.200143 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:30.200146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:30.200159 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:30.200179 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:30.200189 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:30.200303 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:30.200390 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 04:40:30.200425 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:40:30.200733 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.199067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:55957 (local address 127.5.222.193:45163)
0809 04:40:30.199217 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:30.199222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:30.199410 (+ 188us) server_negotiation.cc:407] Connection header received
0809 04:40:30.199486 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:30.199488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:30.199517 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:30.199606 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:30.200315 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:30.200319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:30.200322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:30.200337 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:30.200364 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:30.200378 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:30.200474 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:30.200566 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 04:40:30.200609 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:40:30.200758 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.198891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:45163 (local address 127.5.222.195:55957)
0809 04:40:30.199288 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:30.199298 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:30.199395 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:30.199625 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:30.199628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:30.199643 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:30.200145 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:40:30.200147 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:30.200158 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:40:30.200184 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:30.200476 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:30.200479 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:30.200482 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 04:40:30.200557 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 04:40:30.200589 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":153,"threads_started":1}
I20260809 04:40:30.201478 11557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1362a0c8e1d342128f31e0f311b4e21d" candidate_uuid: "5c5635b8592e4950b6f032192d355f94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "868b04d2561040e9bd4f4fa78c7a569f" is_pre_election: true
I20260809 04:40:30.201478 11707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1362a0c8e1d342128f31e0f311b4e21d" candidate_uuid: "5c5635b8592e4950b6f032192d355f94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "167ff7950c5248d9ab7f8d2694e880b8" is_pre_election: true
I20260809 04:40:30.201789 11557 raft_consensus.cc:2454] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c5635b8592e4950b6f032192d355f94 in term 0.
I20260809 04:40:30.202054 11707 raft_consensus.cc:2454] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c5635b8592e4950b6f032192d355f94 in term 0.
I20260809 04:40:30.202273 11780 leader_election.cc:302] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c5635b8592e4950b6f032192d355f94, 868b04d2561040e9bd4f4fa78c7a569f; no voters:
I20260809 04:40:30.202558 11932 raft_consensus.cc:2781] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:30.202637 11932 raft_consensus.cc:482] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:30.202697 11932 raft_consensus.cc:3037] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:30.249764 11932 raft_consensus.cc:504] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167ff7950c5248d9ab7f8d2694e880b8" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 41793 } } peers { permanent_uuid: "5c5635b8592e4950b6f032192d355f94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 37035 } } peers { permanent_uuid: "868b04d2561040e9bd4f4fa78c7a569f" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 45163 } }
I20260809 04:40:30.250346 11932 leader_election.cc:288] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [CANDIDATE]: Term 1 election: Requested vote from peers 167ff7950c5248d9ab7f8d2694e880b8 (127.5.222.194:41793), 868b04d2561040e9bd4f4fa78c7a569f (127.5.222.193:45163)
I20260809 04:40:30.250766 11707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1362a0c8e1d342128f31e0f311b4e21d" candidate_uuid: "5c5635b8592e4950b6f032192d355f94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "167ff7950c5248d9ab7f8d2694e880b8"
I20260809 04:40:30.250831 11557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1362a0c8e1d342128f31e0f311b4e21d" candidate_uuid: "5c5635b8592e4950b6f032192d355f94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "868b04d2561040e9bd4f4fa78c7a569f"
I20260809 04:40:30.250922 11707 raft_consensus.cc:3037] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:30.250957 11557 raft_consensus.cc:3037] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:30.252372 11557 raft_consensus.cc:2454] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c5635b8592e4950b6f032192d355f94 in term 1.
I20260809 04:40:30.252372 11707 raft_consensus.cc:2454] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c5635b8592e4950b6f032192d355f94 in term 1.
I20260809 04:40:30.252787 11780 leader_election.cc:302] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c5635b8592e4950b6f032192d355f94, 868b04d2561040e9bd4f4fa78c7a569f; no voters:
I20260809 04:40:30.253068 11932 raft_consensus.cc:2781] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:30.253355 11932 raft_consensus.cc:686] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 1 LEADER]: Becoming Leader. State: Replica: 5c5635b8592e4950b6f032192d355f94, State: Running, Role: LEADER
I20260809 04:40:30.253765 11932 consensus_queue.cc:237] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167ff7950c5248d9ab7f8d2694e880b8" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 41793 } } peers { permanent_uuid: "5c5635b8592e4950b6f032192d355f94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 37035 } } peers { permanent_uuid: "868b04d2561040e9bd4f4fa78c7a569f" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 45163 } }
I20260809 04:40:30.257072 11383 catalog_manager.cc:5495] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c5635b8592e4950b6f032192d355f94 (127.5.222.195). New cstate: current_term: 1 leader_uuid: "5c5635b8592e4950b6f032192d355f94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167ff7950c5248d9ab7f8d2694e880b8" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 41793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c5635b8592e4950b6f032192d355f94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 37035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "868b04d2561040e9bd4f4fa78c7a569f" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 45163 } health_report { overall_health: UNKNOWN } } }
W20260809 04:40:30.277828 11905 tablet.cc:2369] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:40:30.303323 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.301828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51942 (local address 127.5.222.195:37035)
0809 04:40:30.301970 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:30.301974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:30.301988 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:40:30.302086 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:30.302088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:30.302145 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:30.302219 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:30.303003 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:30.303006 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:30.303008 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:30.303018 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:30.303037 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:30.303044 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:30.303119 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:30.303186 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 04:40:30.303219 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:40:30.309111 11557 raft_consensus.cc:1261] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f [term 1 FOLLOWER]: Refusing update from remote peer 5c5635b8592e4950b6f032192d355f94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:30.309217 11707 raft_consensus.cc:1261] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8 [term 1 FOLLOWER]: Refusing update from remote peer 5c5635b8592e4950b6f032192d355f94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:30.309988 11934 consensus_queue.cc:1035] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "868b04d2561040e9bd4f4fa78c7a569f" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 45163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:30.310218 11932 consensus_queue.cc:1035] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "167ff7950c5248d9ab7f8d2694e880b8" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 41793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:30.317305 11937 mvcc.cc:205] Tried to move back new op lower bound from 7316481762538930176 to 7316481762322632704. Current Snapshot: MvccSnapshot[applied={T|T < 7316481762538930176}]
I20260809 04:40:30.317942 11938 mvcc.cc:205] Tried to move back new op lower bound from 7316481762538930176 to 7316481762322632704. Current Snapshot: MvccSnapshot[applied={T|T < 7316481762538930176}]
I20260809 04:40:30.320390 11940 mvcc.cc:205] Tried to move back new op lower bound from 7316481762538930176 to 7316481762322632704. Current Snapshot: MvccSnapshot[applied={T|T < 7316481762538930176}]
I20260809 04:40:30.326406 11384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:30.326612 11384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:30.329046 11384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 303df4f3990a46018cc2d327fc4b928a: Sending DeleteTablet for 3 replicas of tablet 1362a0c8e1d342128f31e0f311b4e21d
I20260809 04:40:30.329824 11834 tablet_service.cc:1514] Processing DeleteTablet for tablet 1362a0c8e1d342128f31e0f311b4e21d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:30 UTC) from {username='slave'} at 127.0.0.1:51938
I20260809 04:40:30.329924 11685 tablet_service.cc:1514] Processing DeleteTablet for tablet 1362a0c8e1d342128f31e0f311b4e21d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:30 UTC) from {username='slave'} at 127.0.0.1:51426
I20260809 04:40:30.329921 11534 tablet_service.cc:1514] Processing DeleteTablet for tablet 1362a0c8e1d342128f31e0f311b4e21d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:30 UTC) from {username='slave'} at 127.0.0.1:40602
I20260809 04:40:30.330339 11951 tablet_replica.cc:330] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94: stopping tablet replica
I20260809 04:40:30.330462 11953 tablet_replica.cc:330] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f: stopping tablet replica
I20260809 04:40:30.330762 11953 raft_consensus.cc:2229] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:30.330763 11951 raft_consensus.cc:2229] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:30.331101 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 11459
I20260809 04:40:30.331240 11953 raft_consensus.cc:2258] T 1362a0c8e1d342128f31e0f311b4e21d P 868b04d2561040e9bd4f4fa78c7a569f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:30.331436 11951 raft_consensus.cc:2258] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:30.335008 11951 ts_tablet_manager.cc:1905] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:30.336758 11952 tablet_replica.cc:330] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8: stopping tablet replica
I20260809 04:40:30.337066 11952 raft_consensus.cc:2229] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:30.337411 11952 raft_consensus.cc:2258] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:30.338901 11952 ts_tablet_manager.cc:1905] T 1362a0c8e1d342128f31e0f311b4e21d P 167ff7950c5248d9ab7f8d2694e880b8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 04:40:30.341255 11365 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:45163 (error 108)
I20260809 04:40:30.341427 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 11614
I20260809 04:40:30.343596 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.343224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:45163 (local address 127.0.0.1:40618)
0809 04:40:30.343400 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:30.343486 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:45163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 04:40:30.343874 11365 catalog_manager.cc:4584] TS 868b04d2561040e9bd4f4fa78c7a569f (127.5.222.193:45163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1362a0c8e1d342128f31e0f311b4e21d: Network error: Client connection negotiation failed: client connection to 127.5.222.193:45163: connect: Connection refused (error 111)
I20260809 04:40:30.357872 11951 ts_tablet_manager.cc:1918] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:30.358027 11951 log.cc:1196] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/1362a0c8e1d342128f31e0f311b4e21d
I20260809 04:40:30.358592 11951 ts_tablet_manager.cc:1939] T 1362a0c8e1d342128f31e0f311b4e21d P 5c5635b8592e4950b6f032192d355f94: Deleting consensus metadata
I20260809 04:40:30.360507 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 11762
I20260809 04:40:30.362599 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.360856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:41793 (local address 127.0.0.1:51436)
0809 04:40:30.362443 (+ 1587us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:30.362501 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.194:41793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1497}
I20260809 04:40:30.376526 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 11348
2026-08-09T04:40:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1164 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 04:40:30.404588 6011 mini_oidc.cc:152] Starting JWKS server
I20260809 04:40:30.408316 6011 webserver.cc:460] Webserver started at https://127.0.0.1:33415/ using document root <none> and password file <none>
I20260809 04:40:30.408569 6011 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 04:40:30.408938 6011 webserver.cc:460] Webserver started at http://127.0.0.1:43671/ using document root <none> and password file <none>
2026-08-09T04:40:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:30Z Disabled control of system clock
I20260809 04:40:30.421274 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:40861
--webserver_interface=127.5.222.254
--webserver_port=0
--builtin_ntp_servers=127.5.222.212:39477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:40861
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33415/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/testchainca.cert with env {}
W20260809 04:40:30.538967 11960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:30.539225 11960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:30.539845 11960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:30.543866 11960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:30.543920 11960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:30.543936 11960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:30.543953 11960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:30.543968 11960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:40:30.543983 11960 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:40:30.543996 11960 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33415/jwks.json
I20260809 04:40:30.549343 11960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:39477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:40861
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:40861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33415/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:30.550336 11960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:30.551277 11960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:30.559928 11968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:30.559958 11971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:30.559935 11967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:30.560156 11960 server_base.cc:1034] running on GCE node
I20260809 04:40:30.560801 11960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:30.561985 11960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:30.563146 11960 hybrid_clock.cc:648] HybridClock initialized: now 1786250430563135 us; error 26 us; skew 500 ppm
I20260809 04:40:30.565635 11960 webserver.cc:460] Webserver started at http://127.5.222.254:41363/ using document root <none> and password file <none>
I20260809 04:40:30.566355 11960 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:30.566471 11960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:30.566694 11960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:30.569520 11960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "6bdcc1c2aadb411e87104ddbfcee27df"
format_stamp: "Formatted at 2026-08-09 04:40:30 on dist-test-slave-vxj2"
server_key: "71d45715794a613b238ae7e18142b232"
server_key_iv: "431068417a33edd75119adef7edd1849"
server_key_version: "encryptionkey@0"
I20260809 04:40:30.570051 11960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "6bdcc1c2aadb411e87104ddbfcee27df"
format_stamp: "Formatted at 2026-08-09 04:40:30 on dist-test-slave-vxj2"
server_key: "71d45715794a613b238ae7e18142b232"
server_key_iv: "431068417a33edd75119adef7edd1849"
server_key_version: "encryptionkey@0"
I20260809 04:40:30.574577 11960 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 04:40:30.577099 11979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:30.578469 11960 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:40:30.578651 11960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "6bdcc1c2aadb411e87104ddbfcee27df"
format_stamp: "Formatted at 2026-08-09 04:40:30 on dist-test-slave-vxj2"
server_key: "71d45715794a613b238ae7e18142b232"
server_key_iv: "431068417a33edd75119adef7edd1849"
server_key_version: "encryptionkey@0"
I20260809 04:40:30.578779 11960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:30.589960 11960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:30.590605 11960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:30.590796 11960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:30.598102 11960 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:40861
I20260809 04:40:30.598165 12034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:40861 every 8 connection(s)
I20260809 04:40:30.599103 11960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:30.602589 12035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:30.608098 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 11960
I20260809 04:40:30.608253 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:30.608543 6011 external_mini_cluster.cc:1428] Setting key 5bfe7d3f53604b1109a0cdcbab689818
I20260809 04:40:30.608752 12035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df: Bootstrap starting.
I20260809 04:40:30.611614 12035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:30.612425 12035 log.cc:824] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:30.614614 12035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df: No bootstrap required, opened a new log
I20260809 04:40:30.618178 12038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.609936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50910 (local address 127.5.222.254:40861)
0809 04:40:30.610756 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:30.610772 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:30.610810 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:40:30.611497 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:30.611516 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:30.611957 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:30.612293 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:30.612733 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:30.613915 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:30.614897 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:30.615355 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:30.615558 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:30.616416 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:30.616422 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:30.616434 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:30.616862 (+ 428us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:30.616919 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:30.616939 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:30.617049 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:30.617192 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 04:40:30.617465 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":186,"threads_started":1}
I20260809 04:40:30.618458 12035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdcc1c2aadb411e87104ddbfcee27df" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40861 } }
I20260809 04:40:30.618759 12035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:30.618834 12035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bdcc1c2aadb411e87104ddbfcee27df, State: Initialized, Role: FOLLOWER
I20260809 04:40:30.619324 12035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdcc1c2aadb411e87104ddbfcee27df" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40861 } }
I20260809 04:40:30.619467 12035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:30.619565 12035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:30.619709 12035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:30.621280 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:40:30.669803 12035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdcc1c2aadb411e87104ddbfcee27df" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40861 } }
I20260809 04:40:30.670356 12035 leader_election.cc:302] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bdcc1c2aadb411e87104ddbfcee27df; no voters:
I20260809 04:40:30.670794 12035 leader_election.cc:288] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:30.671020 12040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:30.671272 12040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [term 1 LEADER]: Becoming Leader. State: Replica: 6bdcc1c2aadb411e87104ddbfcee27df, State: Running, Role: LEADER
I20260809 04:40:30.671677 12040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdcc1c2aadb411e87104ddbfcee27df" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40861 } }
I20260809 04:40:30.672340 12035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:30.673631 12042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bdcc1c2aadb411e87104ddbfcee27df. Latest consensus state: current_term: 1 leader_uuid: "6bdcc1c2aadb411e87104ddbfcee27df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdcc1c2aadb411e87104ddbfcee27df" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40861 } } }
I20260809 04:40:30.673789 12042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:30.674201 12041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bdcc1c2aadb411e87104ddbfcee27df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdcc1c2aadb411e87104ddbfcee27df" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40861 } } }
I20260809 04:40:30.674329 12041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:30.674893 12049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:30.678396 12049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:30.684481 12049 catalog_manager.cc:1347] Generated new cluster ID: 91fe80383b7249a9b2c2be28c948a98a
I20260809 04:40:30.684561 12049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:30.696434 12049 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:30.697480 12049 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:30.704113 12049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bdcc1c2aadb411e87104ddbfcee27df: Generated new TSK 0
I20260809 04:40:30.704838 12049 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 04:40:30.740124 12069 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 04:40:30.740291 12069 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 04:40:30.740821 12038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.722830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50920 (local address 127.5.222.254:40861)
0809 04:40:30.722966 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:30.722971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:30.723133 (+ 162us) server_negotiation.cc:407] Connection header received
0809 04:40:30.723329 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:30.723332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:30.723392 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:30.723507 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:40:30.723733 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:30.724592 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:30.725609 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:30.725967 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:30.726065 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:30.726414 (+ 349us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:40:30.740390 (+ 13976us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 04:40:30.740677 (+ 287us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50920: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 04:40:30.740907 12038 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50920: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 04:40:30.741997 12068 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:30.722704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40861 (local address 127.0.0.1:50920)
0809 04:40:30.723047 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:30.723060 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:30.723166 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:30.723524 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:30.723526 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:30.723533 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:40:30.723629 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:30.723633 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:30.724772 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:30.724776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:30.725482 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:30.725491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:30.726100 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:30.726103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:30.726170 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:30.726217 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:40:30.740626 (+ 14409us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 04:40:30.740828 (+ 202us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.222.254:40861: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":115,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.222.254:40861: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33415/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 04:40:30.742781 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 11960
2026-08-09T04:40:30Z chronyd exiting
I20260809 04:40:30.782615 6011 test_util.cc:175] -----------------------------------------------
I20260809 04:40:30.782711 6011 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250403492198-6011-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (380 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 04:40:30.783842 6011 mini_oidc.cc:152] Starting JWKS server
I20260809 04:40:30.784785 6011 webserver.cc:460] Webserver started at https://127.0.0.1:38231/ using document root <none> and password file <none>
I20260809 04:40:30.784904 6011 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 04:40:30.785230 6011 webserver.cc:460] Webserver started at http://127.0.0.1:34989/ using document root <none> and password file <none>
2026-08-09T04:40:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:30Z Disabled control of system clock
I20260809 04:40:30.795684 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:37289
--webserver_interface=127.5.222.254
--webserver_port=0
--builtin_ntp_servers=127.5.222.212:40507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:37289
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38231/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/testchainca.cert with env {}
W20260809 04:40:30.915073 12076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:30.915401 12076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:30.915448 12076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:30.919764 12076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:30.919839 12076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:30.919858 12076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:30.919878 12076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:30.919898 12076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:40:30.919916 12076 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:40:30.919934 12076 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38231/jwks.json
I20260809 04:40:30.925679 12076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:37289
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:37289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38231/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:30.926798 12076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:30.927802 12076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:30.937235 12082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:30.937199 12084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:30.937199 12081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:30.937631 12076 server_base.cc:1034] running on GCE node
I20260809 04:40:30.938300 12076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:30.939623 12076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:30.940807 12076 hybrid_clock.cc:648] HybridClock initialized: now 1786250430940781 us; error 41 us; skew 500 ppm
I20260809 04:40:30.942790 12076 webserver.cc:460] Webserver started at http://127.5.222.254:34837/ using document root <none> and password file <none>
I20260809 04:40:30.943440 12076 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:30.943532 12076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:30.943763 12076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:30.946045 12076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "c15d4de322dd4b40a9a492f4964c0431"
format_stamp: "Formatted at 2026-08-09 04:40:30 on dist-test-slave-vxj2"
server_key: "9c25e9b3c42feadf80c7e697a6e25a09"
server_key_iv: "702e271b116681326b450eb40fc1d5cf"
server_key_version: "encryptionkey@0"
I20260809 04:40:30.946641 12076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "c15d4de322dd4b40a9a492f4964c0431"
format_stamp: "Formatted at 2026-08-09 04:40:30 on dist-test-slave-vxj2"
server_key: "9c25e9b3c42feadf80c7e697a6e25a09"
server_key_iv: "702e271b116681326b450eb40fc1d5cf"
server_key_version: "encryptionkey@0"
I20260809 04:40:30.950428 12076 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:40:30.952732 12090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:30.954000 12076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:30.954149 12076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "c15d4de322dd4b40a9a492f4964c0431"
format_stamp: "Formatted at 2026-08-09 04:40:30 on dist-test-slave-vxj2"
server_key: "9c25e9b3c42feadf80c7e697a6e25a09"
server_key_iv: "702e271b116681326b450eb40fc1d5cf"
server_key_version: "encryptionkey@0"
I20260809 04:40:30.954262 12076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:30.980266 12076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:30.981564 12076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:30.981900 12076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:30.989712 12076 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:37289
I20260809 04:40:30.989724 12149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:37289 every 8 connection(s)
I20260809 04:40:30.990749 12076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:30.992005 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 12076
I20260809 04:40:30.992115 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:30.992475 6011 external_mini_cluster.cc:1428] Setting key b60fc399ee05c0f5aaedccbd8cc87023
I20260809 04:40:30.994385 12150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:31.001261 12150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431: Bootstrap starting.
I20260809 04:40:31.003322 12153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:30.993967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.5.222.254:37289)
0809 04:40:30.994678 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:30.994699 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:30.994742 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:40:30.995622 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:30.995644 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:30.996127 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:30.996504 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:30.996943 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:30.998472 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:30.999408 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.000116 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.000300 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:31.001177 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:31.001185 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:31.001200 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:31.001749 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:31.001819 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:31.001846 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:31.001947 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:31.002097 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 04:40:31.002374 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":120,"threads_started":1}
I20260809 04:40:31.004029 12150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:31.004951 12150 log.cc:824] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:31.006706 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:40:31.007267 12150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431: No bootstrap required, opened a new log
I20260809 04:40:31.011426 12150 raft_consensus.cc:348] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c15d4de322dd4b40a9a492f4964c0431" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37289 } }
I20260809 04:40:31.011689 12150 raft_consensus.cc:374] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:31.011781 12150 raft_consensus.cc:729] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15d4de322dd4b40a9a492f4964c0431, State: Initialized, Role: FOLLOWER
I20260809 04:40:31.012219 12150 consensus_queue.cc:260] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c15d4de322dd4b40a9a492f4964c0431" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37289 } }
I20260809 04:40:31.012379 12150 raft_consensus.cc:388] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:31.012476 12150 raft_consensus.cc:482] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:31.012598 12150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:31.013648 12150 raft_consensus.cc:504] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c15d4de322dd4b40a9a492f4964c0431" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37289 } }
I20260809 04:40:31.013998 12150 leader_election.cc:302] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c15d4de322dd4b40a9a492f4964c0431; no voters:
I20260809 04:40:31.014314 12150 leader_election.cc:288] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:31.014463 12155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:31.014710 12155 raft_consensus.cc:686] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [term 1 LEADER]: Becoming Leader. State: Replica: c15d4de322dd4b40a9a492f4964c0431, State: Running, Role: LEADER
I20260809 04:40:31.015014 12155 consensus_queue.cc:237] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c15d4de322dd4b40a9a492f4964c0431" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37289 } }
I20260809 04:40:31.015570 12150 sys_catalog.cc:564] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:31.016670 12157 sys_catalog.cc:455] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c15d4de322dd4b40a9a492f4964c0431. Latest consensus state: current_term: 1 leader_uuid: "c15d4de322dd4b40a9a492f4964c0431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c15d4de322dd4b40a9a492f4964c0431" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37289 } } }
I20260809 04:40:31.016798 12157 sys_catalog.cc:458] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:31.017112 12156 sys_catalog.cc:455] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c15d4de322dd4b40a9a492f4964c0431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c15d4de322dd4b40a9a492f4964c0431" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37289 } } }
I20260809 04:40:31.017239 12156 sys_catalog.cc:458] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:31.018141 12166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:31.021085 12166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:31.026593 12166 catalog_manager.cc:1347] Generated new cluster ID: 2d0767fa5ce84512a846d8c08475a137
I20260809 04:40:31.026659 12166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:31.047434 12166 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:31.048323 12166 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:31.052553 12166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c15d4de322dd4b40a9a492f4964c0431: Generated new TSK 0
I20260809 04:40:31.053220 12166 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 04:40:31.088531 12182 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 04:40:31.088647 12182 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 04:40:31.089425 12153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:31.070739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32986 (local address 127.5.222.254:37289)
0809 04:40:31.070860 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:31.070865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:31.071105 (+ 240us) server_negotiation.cc:407] Connection header received
0809 04:40:31.071244 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:31.071247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:31.071303 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:31.071424 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:40:31.071620 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.072429 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.073359 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.073704 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.073808 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:31.074130 (+ 322us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:40:31.088938 (+ 14808us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38231/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 04:40:31.089265 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32986: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38231/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 04:40:31.089499 12153 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32986: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38231/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 04:40:31.089579 12181 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:31.070688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37289 (local address 127.0.0.1:32986)
0809 04:40:31.071010 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:31.071023 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:31.071122 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:31.071420 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:31.071422 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:31.071429 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:40:31.071519 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:31.071523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.072541 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.072544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:31.073247 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:31.073257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.073852 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.073856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:31.073928 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:31.073951 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:40:31.089202 (+ 15251us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38231/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 04:40:31.089375 (+ 173us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.222.254:37289: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38231/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":152,"threads_started":1}
I20260809 04:40:31.090342 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 12076
2026-08-09T04:40:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (348 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 04:40:31.132313 6011 mini_oidc.cc:152] Starting JWKS server
I20260809 04:40:31.133249 6011 webserver.cc:460] Webserver started at https://127.0.0.1:33099/ using document root <none> and password file <none>
I20260809 04:40:31.133335 6011 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 04:40:31.133630 6011 webserver.cc:460] Webserver started at http://127.0.0.1:44759/ using document root <none> and password file <none>
2026-08-09T04:40:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:31Z Disabled control of system clock
I20260809 04:40:31.143096 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:43195
--webserver_interface=127.5.222.254
--webserver_port=0
--builtin_ntp_servers=127.5.222.212:45733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:43195
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33099/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 04:40:31.252730 12188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:31.252991 12188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:31.253029 12188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:31.256965 12188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:31.257019 12188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:31.257035 12188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:31.257085 12188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:31.257117 12188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:40:31.257143 12188 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:40:31.257189 12188 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33099/jwks.json
I20260809 04:40:31.262157 12188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:43195
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:43195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33099/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:31.263235 12188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:31.264611 12188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:31.273242 12195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:31.273238 12197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:31.273327 12188 server_base.cc:1034] running on GCE node
W20260809 04:40:31.273242 12194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:31.274060 12188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:31.275184 12188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:31.276381 12188 hybrid_clock.cc:648] HybridClock initialized: now 1786250431276360 us; error 32 us; skew 500 ppm
I20260809 04:40:31.278354 12188 webserver.cc:460] Webserver started at http://127.5.222.254:34875/ using document root <none> and password file <none>
I20260809 04:40:31.278965 12188 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:31.279063 12188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:31.279304 12188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:31.281462 12188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "e8bfc122ee434671affd648f555e66a3"
format_stamp: "Formatted at 2026-08-09 04:40:31 on dist-test-slave-vxj2"
server_key: "c2a4c67b28f9a648a8a6b4d2226c2a17"
server_key_iv: "7f5c7c04848e14ba8f4ea012519a7251"
server_key_version: "encryptionkey@0"
I20260809 04:40:31.282055 12188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "e8bfc122ee434671affd648f555e66a3"
format_stamp: "Formatted at 2026-08-09 04:40:31 on dist-test-slave-vxj2"
server_key: "c2a4c67b28f9a648a8a6b4d2226c2a17"
server_key_iv: "7f5c7c04848e14ba8f4ea012519a7251"
server_key_version: "encryptionkey@0"
I20260809 04:40:31.285897 12188 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:40:31.288264 12205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:31.289544 12188 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 04:40:31.289752 12188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "e8bfc122ee434671affd648f555e66a3"
format_stamp: "Formatted at 2026-08-09 04:40:31 on dist-test-slave-vxj2"
server_key: "c2a4c67b28f9a648a8a6b4d2226c2a17"
server_key_iv: "7f5c7c04848e14ba8f4ea012519a7251"
server_key_version: "encryptionkey@0"
I20260809 04:40:31.289891 12188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:31.303570 12188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:31.304231 12188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:31.304426 12188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:31.311596 12188 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:43195
I20260809 04:40:31.311630 12262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:43195 every 8 connection(s)
I20260809 04:40:31.312515 12188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:31.315928 12264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:31.318596 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 12188
I20260809 04:40:31.318707 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:31.319036 6011 external_mini_cluster.cc:1428] Setting key e88eec5102d38c62828c9ef80846003d
I20260809 04:40:31.321751 12264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3: Bootstrap starting.
I20260809 04:40:31.324157 12264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:31.325078 12264 log.cc:824] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:31.327263 12264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3: No bootstrap required, opened a new log
I20260809 04:40:31.328696 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:31.320479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49886 (local address 127.5.222.254:43195)
0809 04:40:31.321145 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:31.321232 (+ 87us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:31.321272 (+ 40us) server_negotiation.cc:407] Connection header received
0809 04:40:31.321972 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:31.321990 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:31.322381 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:31.322686 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:31.323099 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.324096 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.325396 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.325901 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.326037 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:31.326884 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:31.326890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:31.326902 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:31.327297 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:31.327422 (+ 125us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:31.327446 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:31.327529 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:31.327694 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 04:40:31.327995 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":159,"threads_started":1}
I20260809 04:40:31.331028 12264 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bfc122ee434671affd648f555e66a3" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43195 } }
I20260809 04:40:31.331240 12264 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:31.331319 12264 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8bfc122ee434671affd648f555e66a3, State: Initialized, Role: FOLLOWER
I20260809 04:40:31.331661 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:40:31.331821 12264 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bfc122ee434671affd648f555e66a3" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43195 } }
I20260809 04:40:31.331985 12264 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:31.332053 12264 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:31.332151 12264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:31.333196 12264 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bfc122ee434671affd648f555e66a3" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43195 } }
I20260809 04:40:31.333580 12264 leader_election.cc:302] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8bfc122ee434671affd648f555e66a3; no voters:
I20260809 04:40:31.333974 12264 leader_election.cc:288] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:31.334053 12270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:31.334358 12270 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [term 1 LEADER]: Becoming Leader. State: Replica: e8bfc122ee434671affd648f555e66a3, State: Running, Role: LEADER
I20260809 04:40:31.334663 12270 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bfc122ee434671affd648f555e66a3" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43195 } }
I20260809 04:40:31.335214 12264 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:31.336308 12273 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8bfc122ee434671affd648f555e66a3. Latest consensus state: current_term: 1 leader_uuid: "e8bfc122ee434671affd648f555e66a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bfc122ee434671affd648f555e66a3" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43195 } } }
I20260809 04:40:31.336458 12273 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:31.336771 12271 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8bfc122ee434671affd648f555e66a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8bfc122ee434671affd648f555e66a3" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43195 } } }
I20260809 04:40:31.336869 12271 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:31.336970 12280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:31.339948 12280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:31.345353 12280 catalog_manager.cc:1347] Generated new cluster ID: ba40b459942846cd816bf5a480348a1b
I20260809 04:40:31.345428 12280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:31.367311 12280 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:31.368227 12280 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:31.377465 12280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8bfc122ee434671affd648f555e66a3: Generated new TSK 0
I20260809 04:40:31.378347 12280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:40:31.398346 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:31.393108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49888 (local address 127.5.222.254:43195)
0809 04:40:31.393233 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:31.393237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:31.393651 (+ 414us) server_negotiation.cc:407] Connection header received
0809 04:40:31.393834 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:31.393839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:31.393893 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:31.394007 (+ 114us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:40:31.394247 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.394810 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.395713 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.395957 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.396071 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:31.396395 (+ 324us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:40:31.397814 (+ 1419us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33099/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 04:40:31.398198 (+ 384us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33099/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 04:40:31.398416 12268 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33099/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 04:40:31.400007 12299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:31.393279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43195 (local address 127.0.0.1:49888)
0809 04:40:31.393549 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:31.393564 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:31.393699 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:31.394004 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:31.394007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:31.394015 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:40:31.394133 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:31.394139 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.394930 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.394933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:31.395590 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:31.395598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.396081 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.396084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:31.396172 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:31.396205 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:40:31.398010 (+ 1805us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33099/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 04:40:31.399843 (+ 1833us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.222.254:43195: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33099/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":1648,"thread_start_us":121,"threads_started":1}
I20260809 04:40:31.400553 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 12188
E20260809 04:40:31.410274 12300 webserver.cc:533] Webserver: sslize failed:
E20260809 04:40:31.410367 12300 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T04:40:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T04:40:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:31Z Disabled control of system clock
I20260809 04:40:31.450942 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:42415
--webserver_interface=127.5.222.254
--webserver_port=0
--builtin_ntp_servers=127.5.222.212:44323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:42415
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/testchainca.cert with env {}
W20260809 04:40:31.560820 12304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:31.561127 12304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:31.561192 12304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:31.565063 12304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:31.565114 12304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:31.565128 12304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:31.565160 12304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:31.565177 12304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:40:31.565192 12304 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:40:31.565207 12304 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 04:40:31.570484 12304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:44323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:42415
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:42415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:31.571447 12304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:31.572376 12304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:31.581059 12313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:31.581055 12315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:31.581135 12304 server_base.cc:1034] running on GCE node
W20260809 04:40:31.581055 12312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:31.581871 12304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:31.582926 12304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:31.584097 12304 hybrid_clock.cc:648] HybridClock initialized: now 1786250431584061 us; error 47 us; skew 500 ppm
I20260809 04:40:31.585989 12304 webserver.cc:460] Webserver started at http://127.5.222.254:40467/ using document root <none> and password file <none>
I20260809 04:40:31.586606 12304 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:31.586668 12304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:31.586925 12304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:31.589087 12304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "3e87178cd7a54cf0944aedb1c07677d7"
format_stamp: "Formatted at 2026-08-09 04:40:31 on dist-test-slave-vxj2"
server_key: "ccc9387e840145777481dd5a2b73389a"
server_key_iv: "540316e10027732b9d18bd0994c4b317"
server_key_version: "encryptionkey@0"
I20260809 04:40:31.589785 12304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "3e87178cd7a54cf0944aedb1c07677d7"
format_stamp: "Formatted at 2026-08-09 04:40:31 on dist-test-slave-vxj2"
server_key: "ccc9387e840145777481dd5a2b73389a"
server_key_iv: "540316e10027732b9d18bd0994c4b317"
server_key_version: "encryptionkey@0"
I20260809 04:40:31.593490 12304 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:40:31.595917 12321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:31.597142 12304 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:31.597338 12304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "3e87178cd7a54cf0944aedb1c07677d7"
format_stamp: "Formatted at 2026-08-09 04:40:31 on dist-test-slave-vxj2"
server_key: "ccc9387e840145777481dd5a2b73389a"
server_key_iv: "540316e10027732b9d18bd0994c4b317"
server_key_version: "encryptionkey@0"
I20260809 04:40:31.597450 12304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:31.621362 12304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:31.622154 12304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:31.622397 12304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:31.629782 12304 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:42415
I20260809 04:40:31.629802 12378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:42415 every 8 connection(s)
I20260809 04:40:31.630831 12304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:31.634024 12379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:31.636711 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 12304
I20260809 04:40:31.636865 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:31.637221 6011 external_mini_cluster.cc:1428] Setting key e6e31254ae2b6f5d5eabf770015912b0
I20260809 04:40:31.639747 12379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7: Bootstrap starting.
I20260809 04:40:31.642213 12379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:31.643091 12379 log.cc:824] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:31.645342 12379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7: No bootstrap required, opened a new log
I20260809 04:40:31.646986 12382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:31.638565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55032 (local address 127.5.222.254:42415)
0809 04:40:31.639197 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:31.639215 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:31.639253 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:40:31.639849 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:31.639898 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:31.640299 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:31.640578 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:31.640923 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.642001 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.643321 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.644061 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.644194 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:31.645085 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:31.645091 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:31.645102 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:40:31.645601 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:31.645660 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:40:31.645706 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:31.645792 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:31.645952 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 04:40:31.646255 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":156,"threads_started":1}
I20260809 04:40:31.649125 12379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e87178cd7a54cf0944aedb1c07677d7" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 42415 } }
I20260809 04:40:31.649447 12379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:31.649507 12379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e87178cd7a54cf0944aedb1c07677d7, State: Initialized, Role: FOLLOWER
I20260809 04:40:31.649889 12379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e87178cd7a54cf0944aedb1c07677d7" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 42415 } }
I20260809 04:40:31.650048 12379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:31.650118 12379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:31.650118 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:40:31.650220 12379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:31.651091 12379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e87178cd7a54cf0944aedb1c07677d7" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 42415 } }
I20260809 04:40:31.651417 12379 leader_election.cc:302] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e87178cd7a54cf0944aedb1c07677d7; no voters:
I20260809 04:40:31.651746 12379 leader_election.cc:288] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:31.651790 12384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:31.652016 12384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [term 1 LEADER]: Becoming Leader. State: Replica: 3e87178cd7a54cf0944aedb1c07677d7, State: Running, Role: LEADER
I20260809 04:40:31.652288 12384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e87178cd7a54cf0944aedb1c07677d7" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 42415 } }
I20260809 04:40:31.652956 12379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:31.653991 12385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e87178cd7a54cf0944aedb1c07677d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e87178cd7a54cf0944aedb1c07677d7" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 42415 } } }
I20260809 04:40:31.654009 12386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e87178cd7a54cf0944aedb1c07677d7. Latest consensus state: current_term: 1 leader_uuid: "3e87178cd7a54cf0944aedb1c07677d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e87178cd7a54cf0944aedb1c07677d7" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 42415 } } }
I20260809 04:40:31.654220 12386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:31.654546 12385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7 [sys.catalog]: This master's current role is: LEADER
W20260809 04:40:31.657490 12405 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:40:31.657552 12405 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:40:31.657629 12397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:31.658304 12397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:31.663467 12397 catalog_manager.cc:1347] Generated new cluster ID: 66945d776fea4f19b0df6079794e8a0c
I20260809 04:40:31.663530 12397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:31.681888 12397 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:31.682801 12397 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:31.697079 12397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e87178cd7a54cf0944aedb1c07677d7: Generated new TSK 0
I20260809 04:40:31.697962 12397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:40:31.798331 12382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:31.707986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55044 (local address 127.5.222.254:42415)
0809 04:40:31.708140 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:31.708145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:31.708378 (+ 233us) server_negotiation.cc:407] Connection header received
0809 04:40:31.708547 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:31.708549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:31.708595 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:31.708680 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:40:31.708994 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.709598 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.710824 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.711039 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.711113 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:31.711484 (+ 371us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:40:31.797812 (+ 86328us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 04:40:31.798139 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55044: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 04:40:31.798419 12382 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55044: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 04:40:31.799352 12415 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:31.707939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:42415 (local address 127.0.0.1:55044)
0809 04:40:31.708270 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:31.708287 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:31.708418 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:31.708707 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:31.708711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:31.708719 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:40:31.708879 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:31.708887 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.709716 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.709720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:31.710705 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:31.710717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:31.711156 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:31.711161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:31.711286 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:31.711317 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:40:31.798908 (+ 87591us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 04:40:31.799127 (+ 219us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.222.254:42415: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":134,"threads_started":1}
I20260809 04:40:31.799965 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 12304
2026-08-09T04:40:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (379 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (123 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (122 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:32 dist-test-slave-vxj2 krb5kdc[12428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:32 dist-test-slave-vxj2 krb5kdc[12428](info): set up 2 sockets
Aug 09 04:40:32 dist-test-slave-vxj2 krb5kdc[12428](info): commencing operation
krb5kdc: starting...
W20260809 04:40:33.143697 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:33 dist-test-slave-vxj2 krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:40:33.267966 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:32785
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:32785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:33.380620 12442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:33.381028 12442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:33.381115 12442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:33.387485 12442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:33.387542 12442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:33.387562 12442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:33.387579 12442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:33.387632 12442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:33.392912 12442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:32785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:32785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:33.393977 12442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:33.394987 12442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:33.403400 12449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:33.403507 12454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:33.403403 12450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:33.403733 12442 server_base.cc:1034] running on GCE node
I20260809 04:40:33.404464 12442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:33.406093 12442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:33.407282 12442 hybrid_clock.cc:648] HybridClock initialized: now 1786250433407250 us; error 50 us; skew 500 ppm
Aug 09 04:40:33 dist-test-slave-vxj2 krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250433, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:33.415419 12442 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:40:33.416463 12442 webserver.cc:460] Webserver started at http://127.5.222.254:40811/ using document root <none> and password file <none>
I20260809 04:40:33.417068 12442 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:33.417161 12442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:33.417446 12442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:33.419572 12442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "b6d66b7689094901851fc5da9e4793f8"
format_stamp: "Formatted at 2026-08-09 04:40:33 on dist-test-slave-vxj2"
server_key: "bb30fd9a5fd2edb5cf7e72f1ef6bbf41"
server_key_iv: "f56e5db6e7aba132480dbe297292bac6"
server_key_version: "encryptionkey@0"
I20260809 04:40:33.420197 12442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "b6d66b7689094901851fc5da9e4793f8"
format_stamp: "Formatted at 2026-08-09 04:40:33 on dist-test-slave-vxj2"
server_key: "bb30fd9a5fd2edb5cf7e72f1ef6bbf41"
server_key_iv: "f56e5db6e7aba132480dbe297292bac6"
server_key_version: "encryptionkey@0"
I20260809 04:40:33.424508 12442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 04:40:33.427083 12462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:33.428421 12442 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:40:33.428562 12442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "b6d66b7689094901851fc5da9e4793f8"
format_stamp: "Formatted at 2026-08-09 04:40:33 on dist-test-slave-vxj2"
server_key: "bb30fd9a5fd2edb5cf7e72f1ef6bbf41"
server_key_iv: "f56e5db6e7aba132480dbe297292bac6"
server_key_version: "encryptionkey@0"
I20260809 04:40:33.428673 12442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:33.440173 12442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:33.444171 12442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:33.444371 12442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:33.453001 12442 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:32785
I20260809 04:40:33.453403 12521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:32785 every 8 connection(s)
I20260809 04:40:33.454166 12442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:33.457523 12522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:33.464318 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 12442
I20260809 04:40:33.464461 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:33.464766 6011 external_mini_cluster.cc:1428] Setting key 911ad7b075f8c79fe55458dbc541956b
I20260809 04:40:33.465502 12522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8: Bootstrap starting.
I20260809 04:40:33.468525 12522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:33.469532 12522 log.cc:824] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:33.471902 12522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8: No bootstrap required, opened a new log
Aug 09 04:40:33 dist-test-slave-vxj2 krb5kdc[12428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:33.477308 12522 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d66b7689094901851fc5da9e4793f8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 32785 } }
I20260809 04:40:33.477612 12522 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:33.477672 12522 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d66b7689094901851fc5da9e4793f8, State: Initialized, Role: FOLLOWER
I20260809 04:40:33.478260 12522 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d66b7689094901851fc5da9e4793f8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 32785 } }
I20260809 04:40:33.478434 12522 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:33.478511 12522 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:33.478619 12522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:33.479542 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:33.466311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52684 (local address 127.5.222.254:32785)
0809 04:40:33.466756 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:33.466780 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:33.466814 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:40:33.467490 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:33.467507 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:33.467828 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:33.468156 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:33.469095 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:33.470402 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:33.471320 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:33.471795 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:33.471963 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:33.474568 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:33.474573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:33.474586 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:33.474613 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:33.476360 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:33.476842 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:33.476847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:33.476851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:33.476966 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:33.477328 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:33.477332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:33.477334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:33.477849 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:33.478077 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:33.478387 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 04:40:33.478707 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":155,"threads_started":1}
I20260809 04:40:33.479916 12522 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d66b7689094901851fc5da9e4793f8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 32785 } }
I20260809 04:40:33.480309 12522 leader_election.cc:302] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6d66b7689094901851fc5da9e4793f8; no voters:
I20260809 04:40:33.480659 12522 leader_election.cc:288] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:33.481032 12530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:33.481269 12530 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [term 1 LEADER]: Becoming Leader. State: Replica: b6d66b7689094901851fc5da9e4793f8, State: Running, Role: LEADER
I20260809 04:40:33.481567 12530 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d66b7689094901851fc5da9e4793f8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 32785 } }
I20260809 04:40:33.482017 12522 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:33.483824 12529 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6d66b7689094901851fc5da9e4793f8. Latest consensus state: current_term: 1 leader_uuid: "b6d66b7689094901851fc5da9e4793f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d66b7689094901851fc5da9e4793f8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 32785 } } }
I20260809 04:40:33.483955 12529 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:33.484329 12545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:33.484314 12531 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6d66b7689094901851fc5da9e4793f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d66b7689094901851fc5da9e4793f8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 32785 } } }
I20260809 04:40:33.485430 12531 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:33.488441 12545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:33.493980 12545 catalog_manager.cc:1347] Generated new cluster ID: 01d99162610f4c89858179c18743eab0
I20260809 04:40:33.494045 12545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:33.514161 12545 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:33.515146 12545 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:33.525336 12545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6d66b7689094901851fc5da9e4793f8: Generated new TSK 0
I20260809 04:40:33.526136 12545 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:40:33.560135 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:32785
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:33.672385 12553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:33.672631 12553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:33.672672 12553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:33.676652 12553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:33.676710 12553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:33.676806 12553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:33.681967 12553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:32785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:33.683075 12553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:33.684208 12553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:33.693723 12561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:33.693845 12560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:33.693737 12563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:33.694067 12553 server_base.cc:1034] running on GCE node
I20260809 04:40:33.694535 12553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:33.695225 12553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:33.696439 12553 hybrid_clock.cc:648] HybridClock initialized: now 1786250433696421 us; error 34 us; skew 500 ppm
Aug 09 04:40:33 dist-test-slave-vxj2 krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250433, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:33.699169 12553 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:40:33.700016 12553 webserver.cc:460] Webserver started at http://127.5.222.193:35349/ using document root <none> and password file <none>
I20260809 04:40:33.700577 12553 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:33.700639 12553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:33.700886 12553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:33.703027 12553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "e8fe402fe1eb4bb2a8ba15871c8618f6"
format_stamp: "Formatted at 2026-08-09 04:40:33 on dist-test-slave-vxj2"
server_key: "fb6a3d1cdfa75879c14941963fc118cc"
server_key_iv: "fbec1ac097860073d2af19e0e13eab3b"
server_key_version: "encryptionkey@0"
I20260809 04:40:33.703604 12553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "e8fe402fe1eb4bb2a8ba15871c8618f6"
format_stamp: "Formatted at 2026-08-09 04:40:33 on dist-test-slave-vxj2"
server_key: "fb6a3d1cdfa75879c14941963fc118cc"
server_key_iv: "fbec1ac097860073d2af19e0e13eab3b"
server_key_version: "encryptionkey@0"
I20260809 04:40:33.707401 12553 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:40:33.709817 12570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:33.711035 12553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:33.711174 12553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "e8fe402fe1eb4bb2a8ba15871c8618f6"
format_stamp: "Formatted at 2026-08-09 04:40:33 on dist-test-slave-vxj2"
server_key: "fb6a3d1cdfa75879c14941963fc118cc"
server_key_iv: "fbec1ac097860073d2af19e0e13eab3b"
server_key_version: "encryptionkey@0"
I20260809 04:40:33.711294 12553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:33.722163 12553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:33.724941 12553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:33.725189 12553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:33.725796 12553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:33.726632 12553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:33.726707 12553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:33.726783 12553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:33.726827 12553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:33.736029 12553 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:43917
I20260809 04:40:33.736053 12699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:43917 every 8 connection(s)
I20260809 04:40:33.736930 12553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
Aug 09 04:40:33 dist-test-slave-vxj2 krb5kdc[12428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250433, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:33.746919 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 12553
I20260809 04:40:33.747062 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:33.747416 6011 external_mini_cluster.cc:1428] Setting key d1401736f58d7253eb636bbc15eb32e6
I20260809 04:40:33.750705 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:33.738744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:47029 (local address 127.5.222.254:32785)
0809 04:40:33.738886 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:33.738890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:33.739744 (+ 854us) server_negotiation.cc:407] Connection header received
0809 04:40:33.741196 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:33.741200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:33.741279 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:33.741366 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:33.742534 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:33.743071 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:33.744212 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:33.744467 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:33.744582 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:33.747033 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:33.747037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:33.747041 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:33.747072 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:33.748664 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:33.749188 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:33.749193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:33.749197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:33.749298 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:33.749657 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:33.749660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:33.749662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:33.749842 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:33.749951 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:33.750415 (+ 464us) server_negotiation.cc:299] Negotiation successful
0809 04:40:33.750535 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:40:33.751453 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:33.739111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:32785 (local address 127.5.222.193:47029)
0809 04:40:33.739588 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:33.739630 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:33.740960 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:33.741550 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:33.741556 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:33.741912 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:33.742378 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:33.742390 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:33.743203 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:33.743206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:33.744078 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:33.744087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:33.744623 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:33.744627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:33.744802 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:33.745379 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:33.745395 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:33.746859 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:33.748795 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:33.748801 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:33.748814 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:33.749070 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:33.749394 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:33.749398 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:33.749400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:33.749552 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:33.749951 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:33.749957 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:33.750186 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 04:40:33.750370 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 04:40:33.750670 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":135,"threads_started":1}
I20260809 04:40:33.752614 12700 heartbeater.cc:347] Connected to a master server at 127.5.222.254:32785
I20260809 04:40:33.752897 12700 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:33.753435 12700 heartbeater.cc:511] Master 127.5.222.254:32785 requested a full tablet report, sending...
I20260809 04:40:33.755102 12473 ts_manager.cc:194] Registered new tserver with Master: e8fe402fe1eb4bb2a8ba15871c8618f6 (127.5.222.193:43917)
I20260809 04:40:33.756731 12473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:47029
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:40:33.776414 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:32785
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:33.887936 12705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:33.888218 12705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:33.888329 12705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:33.892621 12705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:33.892706 12705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:33.892791 12705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:33.897914 12705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:32785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:33.898912 12705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:33.899915 12705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:33.909538 12710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:33.909602 12705 server_base.cc:1034] running on GCE node
W20260809 04:40:33.909524 12713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:33.909524 12711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:33.910148 12705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:33.910773 12705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:33.911958 12705 hybrid_clock.cc:648] HybridClock initialized: now 1786250433911936 us; error 31 us; skew 500 ppm
Aug 09 04:40:33 dist-test-slave-vxj2 krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250433, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:33.914897 12705 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:40:33.915791 12705 webserver.cc:460] Webserver started at http://127.5.222.194:44713/ using document root <none> and password file <none>
I20260809 04:40:33.916298 12705 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:33.916356 12705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:33.916527 12705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:33.918603 12705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "90dc83ca958b454297f3a8e8b9b870ce"
format_stamp: "Formatted at 2026-08-09 04:40:33 on dist-test-slave-vxj2"
server_key: "e8ad7c3c42a68a6f264784fb59dc3765"
server_key_iv: "217559086260a65d2189fe0ea086fc97"
server_key_version: "encryptionkey@0"
I20260809 04:40:33.919191 12705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "90dc83ca958b454297f3a8e8b9b870ce"
format_stamp: "Formatted at 2026-08-09 04:40:33 on dist-test-slave-vxj2"
server_key: "e8ad7c3c42a68a6f264784fb59dc3765"
server_key_iv: "217559086260a65d2189fe0ea086fc97"
server_key_version: "encryptionkey@0"
I20260809 04:40:33.923082 12705 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:40:33.925417 12720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:33.926671 12705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:33.926764 12705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "90dc83ca958b454297f3a8e8b9b870ce"
format_stamp: "Formatted at 2026-08-09 04:40:33 on dist-test-slave-vxj2"
server_key: "e8ad7c3c42a68a6f264784fb59dc3765"
server_key_iv: "217559086260a65d2189fe0ea086fc97"
server_key_version: "encryptionkey@0"
I20260809 04:40:33.926896 12705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:33.946774 12705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:33.950143 12705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:33.950341 12705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:33.950922 12705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:33.951737 12705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:33.951783 12705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:33.951846 12705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:33.951879 12705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:33.960872 12705 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:41329
I20260809 04:40:33.960898 12855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:41329 every 8 connection(s)
I20260809 04:40:33.961776 12705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:33.962922 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 12705
I20260809 04:40:33.963040 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:33.963346 6011 external_mini_cluster.cc:1428] Setting key c2875616688ca0450c6daed173f61d4f
Aug 09 04:40:33 dist-test-slave-vxj2 krb5kdc[12428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250433, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
I20260809 04:40:33.976843 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:33.963658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:46017 (local address 127.5.222.254:32785)
0809 04:40:33.963821 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:33.963826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:33.964749 (+ 923us) server_negotiation.cc:407] Connection header received
0809 04:40:33.966824 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:33.966827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:33.966900 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:33.967007 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:33.968195 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:33.969082 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:33.970252 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:33.970694 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:33.970842 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:33.973238 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:33.973243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:33.973247 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:33.973279 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:33.974784 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:33.975323 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:33.975326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:33.975327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:33.975405 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:33.975728 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:33.975730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:33.975731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:33.975912 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:33.975992 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:33.976537 (+ 545us) server_negotiation.cc:299] Negotiation successful
0809 04:40:33.976633 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:40:33.977753 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:33.964094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:32785 (local address 127.5.222.194:46017)
0809 04:40:33.964577 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:33.964642 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:33.966585 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:33.967214 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:33.967221 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:33.967576 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:33.968044 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:33.968057 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:33.969246 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:33.969249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:33.970113 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:33.970122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:33.970859 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:33.970862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:33.971008 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:33.971598 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:33.971614 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:33.973035 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:33.974921 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:33.974934 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:33.974953 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:33.975205 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:33.975510 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:33.975513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:33.975514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:33.975643 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:33.976004 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:33.976008 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:33.976314 (+ 306us) client_negotiation.cc:769] Sending connection context
0809 04:40:33.976475 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 04:40:33.976782 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":114,"threads_started":1}
I20260809 04:40:33.979163 12857 heartbeater.cc:347] Connected to a master server at 127.5.222.254:32785
I20260809 04:40:33.979446 12857 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:33.980036 12857 heartbeater.cc:511] Master 127.5.222.254:32785 requested a full tablet report, sending...
I20260809 04:40:33.981145 12473 ts_manager.cc:194] Registered new tserver with Master: 90dc83ca958b454297f3a8e8b9b870ce (127.5.222.194:41329)
I20260809 04:40:33.981805 12473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:46017
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:40:33.995873 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:32785
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:34.108184 12862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:34.108500 12862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:34.108546 12862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:34.112640 12862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:34.112699 12862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:34.112756 12862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:34.118131 12862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:32785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:34.119089 12862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:34.120050 12862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:34.129814 12868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:34.129875 12870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:34.129976 12862 server_base.cc:1034] running on GCE node
W20260809 04:40:34.130169 12867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:34.130582 12862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:34.131271 12862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:34.132424 12862 hybrid_clock.cc:648] HybridClock initialized: now 1786250434132403 us; error 32 us; skew 500 ppm
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250434, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:34.135139 12862 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:40:34.135890 12862 webserver.cc:460] Webserver started at http://127.5.222.195:34027/ using document root <none> and password file <none>
I20260809 04:40:34.136386 12862 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:34.136445 12862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:34.136602 12862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:34.138626 12862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "afac9ab4a44c48cbb80fb90c78b9b7fb"
format_stamp: "Formatted at 2026-08-09 04:40:34 on dist-test-slave-vxj2"
server_key: "c9d028407188809b839554de1f3cec65"
server_key_iv: "e3a8ee90c3c20985e05f0eb864e8375e"
server_key_version: "encryptionkey@0"
I20260809 04:40:34.139180 12862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "afac9ab4a44c48cbb80fb90c78b9b7fb"
format_stamp: "Formatted at 2026-08-09 04:40:34 on dist-test-slave-vxj2"
server_key: "c9d028407188809b839554de1f3cec65"
server_key_iv: "e3a8ee90c3c20985e05f0eb864e8375e"
server_key_version: "encryptionkey@0"
I20260809 04:40:34.142772 12862 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:40:34.144986 12879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:34.146207 12862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:34.146334 12862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "afac9ab4a44c48cbb80fb90c78b9b7fb"
format_stamp: "Formatted at 2026-08-09 04:40:34 on dist-test-slave-vxj2"
server_key: "c9d028407188809b839554de1f3cec65"
server_key_iv: "e3a8ee90c3c20985e05f0eb864e8375e"
server_key_version: "encryptionkey@0"
I20260809 04:40:34.146445 12862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:34.158843 12862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:34.162016 12862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:34.162215 12862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:34.162765 12862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:34.163544 12862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:34.163592 12862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:34.163658 12862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:34.163695 12862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:34.172672 12862 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:35953
I20260809 04:40:34.172695 13006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:35953 every 8 connection(s)
I20260809 04:40:34.173625 12862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:34.182591 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 12862
I20260809 04:40:34.182762 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[12428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250434, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:34.183099 6011 external_mini_cluster.cc:1428] Setting key e3fa026a5ba2aab1a9bf7ef43516c64f
I20260809 04:40:34.187196 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:34.175473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:34857 (local address 127.5.222.254:32785)
0809 04:40:34.175629 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:34.175634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:34.176507 (+ 873us) server_negotiation.cc:407] Connection header received
0809 04:40:34.177899 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:34.177902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:34.177951 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:34.178027 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:34.179122 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:34.179655 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:34.180718 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:34.180918 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:34.180993 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:34.183755 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:34.183759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:34.183761 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:34.183786 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:34.185441 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:34.185900 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:34.185903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:34.185904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:34.185982 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:34.186231 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:34.186233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:34.186234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:34.186455 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:34.186561 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:34.186977 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 04:40:34.187071 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:40:34.187991 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:34.175884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:32785 (local address 127.5.222.195:34857)
0809 04:40:34.176337 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:34.176379 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:34.177705 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:34.178184 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:34.178190 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:34.178540 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:34.178995 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:34.179007 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:34.179761 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:34.179764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:34.180612 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:34.180621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:34.181043 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:34.181045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:34.181258 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:34.181846 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:34.181861 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:34.183529 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:34.185564 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:34.185569 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:34.185580 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:34.185817 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:34.186042 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:34.186043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:34.186045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:34.186156 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:34.186549 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:34.186554 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:34.186770 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 04:40:34.186926 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 04:40:34.187168 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":112,"threads_started":1}
I20260809 04:40:34.189244 13007 heartbeater.cc:347] Connected to a master server at 127.5.222.254:32785
I20260809 04:40:34.189498 13007 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:34.190011 13007 heartbeater.cc:511] Master 127.5.222.254:32785 requested a full tablet report, sending...
I20260809 04:40:34.191062 12473 ts_manager.cc:194] Registered new tserver with Master: afac9ab4a44c48cbb80fb90c78b9b7fb (127.5.222.195:35953)
I20260809 04:40:34.191623 12473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:34857
I20260809 04:40:34.197062 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/test-admin.keytab.
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:34.214008 6011 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 04:40:34.216832 13014 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:34.215430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59320 (local address 127.5.222.193:43917)
0809 04:40:34.215733 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:34.215747 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:34.215772 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:40:34.215821 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:34.215826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:34.216008 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:34.216178 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:34.216658 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59320: BlockingRecv error: recv got EOF from 127.0.0.1:59320 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":109,"threads_started":1}
W20260809 04:40:34.217137 6011 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/krb5cc' of type FILE
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:34.218302 6011 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:34.219245 6011 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[12428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.193@KRBTEST.COM
I20260809 04:40:34.228588 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:34.220055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59334 (local address 127.5.222.193:43917)
0809 04:40:34.220193 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:34.220196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:34.220210 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:40:34.220323 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:34.220325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:34.220360 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:34.220435 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:34.220949 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:34.221786 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:34.222585 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:34.222943 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:34.223024 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:34.225377 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:34.225386 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:34.225398 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:34.225429 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:34.226889 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:34.227286 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:34.227290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:34.227291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:34.227364 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:34.227611 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:34.227613 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:34.227614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:34.227842 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:34.228016 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:34.228241 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 04:40:34.228432 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 04:40:34.232456 13014 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:34.231457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59346 (local address 127.5.222.193:43917)
0809 04:40:34.231580 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:34.231584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:34.231681 (+ 97us) server_negotiation.cc:407] Connection header received
0809 04:40:34.231827 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:34.231829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:34.231861 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:34.232010 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:34.232355 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59346: BlockingRecv error: recv got EOF from 127.0.0.1:59346 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 04:40:34.232628 6011 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/krb5cc' of type FILE
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:34.233719 6011 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:34.234589 6011 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[12428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.193@KRBTEST.COM
I20260809 04:40:34.243098 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:34.235421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59358 (local address 127.5.222.193:43917)
0809 04:40:34.235542 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:34.235548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:34.235712 (+ 164us) server_negotiation.cc:407] Connection header received
0809 04:40:34.235814 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:34.235816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:34.235850 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:34.235916 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:34.236372 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:34.236921 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:34.237806 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:34.238029 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:34.238114 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:34.240169 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:34.240172 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:34.240174 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:34.240195 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:34.241598 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:34.241990 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:34.241993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:34.241994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:34.242071 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:34.242299 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:34.242301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:34.242302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:34.242454 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:34.242553 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:34.242796 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 04:40:34.242965 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 04:40:34.246107 13014 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:34.244849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.5.222.193:43917)
0809 04:40:34.244988 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:34.244992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:34.245092 (+ 100us) server_negotiation.cc:407] Connection header received
0809 04:40:34.245285 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:34.245287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:34.245331 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:34.245415 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:34.245988 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59370: BlockingRecv error: recv got EOF from 127.0.0.1:59370 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 04:40:34.246209 6011 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250403492198-6011-0/krb5kdc/krb5cc' of type FILE
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:34.247253 6011 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:34.248181 6011 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[12428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.193@KRBTEST.COM
I20260809 04:40:34.256819 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:34.249026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59384 (local address 127.5.222.193:43917)
0809 04:40:34.249158 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:34.249186 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:34.249299 (+ 113us) server_negotiation.cc:407] Connection header received
0809 04:40:34.249446 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:34.249449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:34.249493 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:34.249580 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:34.250128 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:34.250610 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:34.251449 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:34.251666 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:34.251770 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:34.253990 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:34.253993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:34.253996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:34.254018 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:34.255224 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:34.255670 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:34.255674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:34.255676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:34.255760 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:34.256028 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:34.256033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:34.256035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:34.256195 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:34.256328 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:34.256562 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 04:40:34.256665 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:40:34.257786 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 12553
I20260809 04:40:34.266618 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 12705
I20260809 04:40:34.274121 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 12862
I20260809 04:40:34.280382 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 12442
2026-08-09T04:40:34Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2235 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[13028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[13028](info): set up 2 sockets
Aug 09 04:40:34 dist-test-slave-vxj2 krb5kdc[13028](info): commencing operation
krb5kdc: starting...
W20260809 04:40:35.372871 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:35 dist-test-slave-vxj2 krb5kdc[13028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.254.keytab.
Entry for principal oryx/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.254.keytab.
I20260809 04:40:35.504873 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:43161
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.254.keytab
--principal=oryx/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:43161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:35.614218 13043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:35.614509 13043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:35.614588 13043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:35.618587 13043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:35.618667 13043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:35.618706 13043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:35.618762 13043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:35.618825 13043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:35.624114 13043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:43161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.254.keytab
--principal=oryx/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:43161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:35.625270 13043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:35.626296 13043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:35.634886 13049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:35.634826 13050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:35.634938 13043 server_base.cc:1034] running on GCE node
W20260809 04:40:35.634876 13052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:35.635610 13043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:35.636651 13043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:35.637806 13043 hybrid_clock.cc:648] HybridClock initialized: now 1786250435637775 us; error 43 us; skew 500 ppm
Aug 09 04:40:35 dist-test-slave-vxj2 krb5kdc[13028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250435, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:35.640539 13043 init.cc:377] Logged in from keytab as oryx/127.5.222.254@KRBTEST.COM (short username oryx)
I20260809 04:40:35.641427 13043 webserver.cc:460] Webserver started at http://127.5.222.254:38305/ using document root <none> and password file <none>
I20260809 04:40:35.641955 13043 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:35.642015 13043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:35.642263 13043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:35.644212 13043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "086656952f8644e4a572db00a1becc1f"
format_stamp: "Formatted at 2026-08-09 04:40:35 on dist-test-slave-vxj2"
server_key: "afa5043426fd286204231a67a63693bf"
server_key_iv: "ad70e31cb00a9161de005ac578e80d0f"
server_key_version: "encryptionkey@0"
I20260809 04:40:35.644785 13043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "086656952f8644e4a572db00a1becc1f"
format_stamp: "Formatted at 2026-08-09 04:40:35 on dist-test-slave-vxj2"
server_key: "afa5043426fd286204231a67a63693bf"
server_key_iv: "ad70e31cb00a9161de005ac578e80d0f"
server_key_version: "encryptionkey@0"
I20260809 04:40:35.648288 13043 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 04:40:35.650579 13062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:35.651695 13043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:40:35.651834 13043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "086656952f8644e4a572db00a1becc1f"
format_stamp: "Formatted at 2026-08-09 04:40:35 on dist-test-slave-vxj2"
server_key: "afa5043426fd286204231a67a63693bf"
server_key_iv: "ad70e31cb00a9161de005ac578e80d0f"
server_key_version: "encryptionkey@0"
I20260809 04:40:35.651955 13043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:35.666880 13043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:35.669538 13043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:35.669732 13043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:35.676343 13043 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:43161
I20260809 04:40:35.676646 13122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:43161 every 8 connection(s)
I20260809 04:40:35.677359 13043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:35.680071 13123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:35.680827 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 13043
I20260809 04:40:35.680990 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:35.681365 6011 external_mini_cluster.cc:1428] Setting key 858f2e1e0cd702482e09304d8c1cb995
I20260809 04:40:35.686815 13123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f: Bootstrap starting.
I20260809 04:40:35.689483 13123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:35.690272 13123 log.cc:824] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:40:35 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:35.692468 13123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f: No bootstrap required, opened a new log
I20260809 04:40:35.695930 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:35.682749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55284 (local address 127.5.222.254:43161)
0809 04:40:35.683187 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:35.683206 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:35.683252 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:40:35.684050 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:35.684075 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:35.684484 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:35.684843 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:35.685597 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:35.687050 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:35.687872 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:35.688567 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:35.688730 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:35.691389 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:35.691436 (+ 47us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:35.691449 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:35.691475 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:35.693217 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:35.693654 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:35.693662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:35.693671 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:35.693780 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:35.694070 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:35.694074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:35.694075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:35.694460 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:35.694655 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:35.694947 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 04:40:35.695230 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":114,"threads_started":1}
I20260809 04:40:35.695971 13123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } }
I20260809 04:40:35.696262 13123 raft_consensus.cc:374] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:35.696331 13123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086656952f8644e4a572db00a1becc1f, State: Initialized, Role: FOLLOWER
I20260809 04:40:35.696776 13123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } }
I20260809 04:40:35.696923 13123 raft_consensus.cc:388] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:35.696990 13123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:35.697095 13123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:35.698057 13123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } }
I20260809 04:40:35.698421 13123 leader_election.cc:302] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 086656952f8644e4a572db00a1becc1f; no voters:
I20260809 04:40:35.698685 13123 leader_election.cc:288] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:35.698812 13129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:35.699100 13129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 1 LEADER]: Becoming Leader. State: Replica: 086656952f8644e4a572db00a1becc1f, State: Running, Role: LEADER
I20260809 04:40:35.699411 13129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } }
I20260809 04:40:35.699884 13123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:35.701040 13131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 086656952f8644e4a572db00a1becc1f. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } }
I20260809 04:40:35.701198 13131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:35.701555 13139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:35.701480 13130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } }
I20260809 04:40:35.702368 13130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:35.704759 13139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:35.710116 13139 catalog_manager.cc:1347] Generated new cluster ID: c2bfbc2767904c37bd7cfea1ce85703e
I20260809 04:40:35.710184 13139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:35.750881 13139 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:35.752065 13139 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:35.765698 13139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f: Generated new TSK 0
I20260809 04:40:35.766624 13139 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab.
Entry for principal oryx/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab.
I20260809 04:40:35.809857 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43161
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab
--principal=oryx/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:35.920847 13152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:35.921243 13152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:35.921311 13152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:35.925416 13152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:35.925472 13152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:35.925529 13152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:35.930579 13152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab
--principal=oryx/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:35.931507 13152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:35.932438 13152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:35.942129 13159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:35.942173 13161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:35.942194 13158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:35.942412 13152 server_base.cc:1034] running on GCE node
I20260809 04:40:35.942775 13152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:35.943413 13152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:35.944615 13152 hybrid_clock.cc:648] HybridClock initialized: now 1786250435944586 us; error 42 us; skew 500 ppm
Aug 09 04:40:35 dist-test-slave-vxj2 krb5kdc[13028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250435, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:35.947535 13152 init.cc:377] Logged in from keytab as oryx/127.5.222.193@KRBTEST.COM (short username oryx)
I20260809 04:40:35.948348 13152 webserver.cc:460] Webserver started at http://127.5.222.193:42013/ using document root <none> and password file <none>
I20260809 04:40:35.948916 13152 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:35.949016 13152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:35.949318 13152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:35.951294 13152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "8750546d7c214df3902dfe9a782667c0"
format_stamp: "Formatted at 2026-08-09 04:40:35 on dist-test-slave-vxj2"
server_key: "924ab39fc2d9a21dea35d579b76b0dcb"
server_key_iv: "af1e8439c95f2fec6282cc4b96f81600"
server_key_version: "encryptionkey@0"
I20260809 04:40:35.951858 13152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "8750546d7c214df3902dfe9a782667c0"
format_stamp: "Formatted at 2026-08-09 04:40:35 on dist-test-slave-vxj2"
server_key: "924ab39fc2d9a21dea35d579b76b0dcb"
server_key_iv: "af1e8439c95f2fec6282cc4b96f81600"
server_key_version: "encryptionkey@0"
I20260809 04:40:35.955616 13152 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 04:40:35.958086 13168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:35.959242 13152 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:35.959373 13152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "8750546d7c214df3902dfe9a782667c0"
format_stamp: "Formatted at 2026-08-09 04:40:35 on dist-test-slave-vxj2"
server_key: "924ab39fc2d9a21dea35d579b76b0dcb"
server_key_iv: "af1e8439c95f2fec6282cc4b96f81600"
server_key_version: "encryptionkey@0"
I20260809 04:40:35.959489 13152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:35.981535 13152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:35.984706 13152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:35.984922 13152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:35.985598 13152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:35.986460 13152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:35.986541 13152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:35.986629 13152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:35.986682 13152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:35.995477 13152 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:38087
I20260809 04:40:35.995499 13299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:38087 every 8 connection(s)
I20260809 04:40:35.996400 13152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:35.997260 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 13152
I20260809 04:40:35.997392 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:35.997700 6011 external_mini_cluster.cc:1428] Setting key b86099b5e8f38837c01fff539d4127e1
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250435, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.5.222.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.194@KRBTEST.COM" created.
I20260809 04:40:36.013064 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:35.998533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:45141 (local address 127.5.222.254:43161)
0809 04:40:35.998713 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:35.998716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:35.999687 (+ 971us) server_negotiation.cc:407] Connection header received
0809 04:40:36.001686 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.001689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.001747 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.001827 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.003182 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.004079 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.005586 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.006065 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.006200 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.009105 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.009108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.009110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.009137 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:36.010701 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.011256 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.011259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.011260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.011350 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.011750 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.011752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.011753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.011946 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:36.012027 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:36.012752 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 04:40:36.012918 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 04:40:36.014139 13302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:35.998916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.193:45141)
0809 04:40:35.999514 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:35.999574 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:36.001476 (+ 1902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:36.001999 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:36.002006 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:36.002505 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:36.003029 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.003041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.004274 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.004277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.005450 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.005461 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.006295 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.006298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.006460 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.007133 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:36.007149 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:36.008836 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:36.010822 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.010827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.010840 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.011165 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.011440 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.011442 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.011444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.011661 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.012066 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:36.012074 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:36.012438 (+ 364us) client_negotiation.cc:769] Sending connection context
0809 04:40:36.012675 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 04:40:36.013052 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":159,"threads_started":1}
I20260809 04:40:36.015488 13300 heartbeater.cc:347] Connected to a master server at 127.5.222.254:43161
I20260809 04:40:36.015776 13300 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:36.016462 13300 heartbeater.cc:511] Master 127.5.222.254:43161 requested a full tablet report, sending...
I20260809 04:40:36.018096 13081 ts_manager.cc:194] Registered new tserver with Master: 8750546d7c214df3902dfe9a782667c0 (127.5.222.193:38087)
I20260809 04:40:36.019214 13081 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:45141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab.
Entry for principal oryx/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab.
I20260809 04:40:36.028554 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43161
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab
--principal=oryx/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:36.136399 13307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:36.136631 13307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:36.136672 13307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:36.140627 13307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:36.140677 13307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:36.140731 13307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:36.145730 13307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab
--principal=oryx/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:36.146728 13307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:36.147686 13307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:36.157084 13314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:36.157084 13313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:36.157085 13316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:36.157363 13307 server_base.cc:1034] running on GCE node
I20260809 04:40:36.157963 13307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:36.158551 13307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:36.159754 13307 hybrid_clock.cc:648] HybridClock initialized: now 1786250436159697 us; error 31 us; skew 500 ppm
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:36.162578 13307 init.cc:377] Logged in from keytab as oryx/127.5.222.194@KRBTEST.COM (short username oryx)
I20260809 04:40:36.163520 13307 webserver.cc:460] Webserver started at http://127.5.222.194:46227/ using document root <none> and password file <none>
I20260809 04:40:36.164124 13307 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:36.164187 13307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:36.164402 13307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:36.166455 13307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "0b660677db014b769e872ce5abb04084"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "c676741c6fd0047e74006798496b8863"
server_key_iv: "95ec0fece0aaa4c0ac1df9c56b54da57"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.167011 13307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "0b660677db014b769e872ce5abb04084"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "c676741c6fd0047e74006798496b8863"
server_key_iv: "95ec0fece0aaa4c0ac1df9c56b54da57"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.170676 13307 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:40:36.173326 13324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:36.174644 13307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:36.174750 13307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "0b660677db014b769e872ce5abb04084"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "c676741c6fd0047e74006798496b8863"
server_key_iv: "95ec0fece0aaa4c0ac1df9c56b54da57"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.174834 13307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:36.185412 13307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:36.188187 13307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:36.188334 13307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:36.188814 13307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:36.189651 13307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:36.189705 13307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:36.189750 13307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:36.189766 13307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:36.198761 13307 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:43927
I20260809 04:40:36.198833 13464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:43927 every 8 connection(s)
I20260809 04:40:36.199625 13307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:36.204810 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 13307
I20260809 04:40:36.204943 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:36.205293 6011 external_mini_cluster.cc:1428] Setting key ec5c5e3645fa2e545e2a4db26341a249
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:36.213958 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.201541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:38091 (local address 127.5.222.254:43161)
0809 04:40:36.201714 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:36.201720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:36.202547 (+ 827us) server_negotiation.cc:407] Connection header received
0809 04:40:36.203914 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.203918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.203980 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.204098 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.205483 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.206050 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.207246 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.207487 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.207606 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.210157 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.210162 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.210164 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.210193 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:36.211765 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.212300 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.212307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.212309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.212408 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.212788 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.212793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.212794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.212989 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:36.213109 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:36.213631 (+ 522us) server_negotiation.cc:299] Negotiation successful
0809 04:40:36.213772 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for oryx/127.5.222.195@KRBTEST.COM; defaulting to no policy
I20260809 04:40:36.214823 13469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.201936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.194:38091)
0809 04:40:36.202400 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:36.202442 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:36.203676 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:36.204270 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:36.204277 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:36.204625 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:36.205261 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.205285 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.206211 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.206215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.207101 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.207110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.207642 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.207646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.207828 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.208452 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:36.208470 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:36.209987 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:36.211895 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.211901 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.211913 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.212181 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.212522 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.212525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.212528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.212671 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.213106 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:36.213113 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:36.213386 (+ 273us) client_negotiation.cc:769] Sending connection context
0809 04:40:36.213585 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 04:40:36.213892 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":113,"threads_started":1}
I20260809 04:40:36.215975 13465 heartbeater.cc:347] Connected to a master server at 127.5.222.254:43161
I20260809 04:40:36.216209 13465 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:36.216673 13465 heartbeater.cc:511] Master 127.5.222.254:43161 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.195@KRBTEST.COM" created.
I20260809 04:40:36.217658 13081 ts_manager.cc:194] Registered new tserver with Master: 0b660677db014b769e872ce5abb04084 (127.5.222.194:43927)
I20260809 04:40:36.218276 13081 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:38091
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab.
Entry for principal oryx/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab.
I20260809 04:40:36.233404 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43161
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab
--principal=oryx/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:36.343999 13472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:36.344231 13472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:36.344272 13472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:36.348243 13472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:36.348294 13472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:36.348351 13472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:36.353703 13472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab
--principal=oryx/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:36.354697 13472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:36.355674 13472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:36.365090 13478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:36.365096 13481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:36.365109 13479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:36.365538 13472 server_base.cc:1034] running on GCE node
I20260809 04:40:36.365908 13472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:36.366537 13472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:36.367754 13472 hybrid_clock.cc:648] HybridClock initialized: now 1786250436367716 us; error 43 us; skew 500 ppm
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:36.370582 13472 init.cc:377] Logged in from keytab as oryx/127.5.222.195@KRBTEST.COM (short username oryx)
I20260809 04:40:36.371411 13472 webserver.cc:460] Webserver started at http://127.5.222.195:40439/ using document root <none> and password file <none>
I20260809 04:40:36.371989 13472 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:36.372049 13472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:36.372262 13472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:36.374233 13472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "1a56bf57806d4996bf4de69813ea10d6"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "e8f45d3a0a9c3a4422e2203e2cd9f32f"
server_key_iv: "163106bc00fdc3bbdd868728b2755c70"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.374779 13472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "1a56bf57806d4996bf4de69813ea10d6"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "e8f45d3a0a9c3a4422e2203e2cd9f32f"
server_key_iv: "163106bc00fdc3bbdd868728b2755c70"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.378489 13472 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 04:40:36.380956 13488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:36.382248 13472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:40:36.382341 13472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "1a56bf57806d4996bf4de69813ea10d6"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "e8f45d3a0a9c3a4422e2203e2cd9f32f"
server_key_iv: "163106bc00fdc3bbdd868728b2755c70"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.382421 13472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:36.411166 13472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:36.414397 13472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:36.414551 13472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:36.415076 13472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:36.415833 13472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:36.415881 13472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:36.415930 13472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:36.415946 13472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:36.424712 13472 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:38385
I20260809 04:40:36.424795 13615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:38385 every 8 connection(s)
I20260809 04:40:36.425698 13472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:36.429749 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 13472
I20260809 04:40:36.429910 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:40:36.430307 6011 external_mini_cluster.cc:1428] Setting key c2de771020b6106e08c80a1406f3d905
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:36.439719 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.427541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:39901 (local address 127.5.222.254:43161)
0809 04:40:36.427689 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:36.427692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:36.428488 (+ 796us) server_negotiation.cc:407] Connection header received
0809 04:40:36.429896 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.429899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.429968 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.430085 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.431630 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.432404 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.433618 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.433862 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.433983 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.436370 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.436374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.436377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.436406 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:36.437672 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.438175 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.438179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.438180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.438276 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.438631 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.438634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.438636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.438839 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:36.438951 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:36.439428 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 04:40:36.439547 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:40:36.440510 13619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.427902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.195:39901)
0809 04:40:36.428326 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:36.428367 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:36.429689 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:36.430308 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:36.430320 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:36.430824 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:36.431461 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.431483 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.432533 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.432537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.433481 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.433491 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.434021 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.434026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.434203 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.434735 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:36.434792 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:36.436200 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:36.437795 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.437801 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.437814 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.438067 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.438370 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.438374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.438376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.438511 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.438952 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:36.438960 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:36.439194 (+ 234us) client_negotiation.cc:769] Sending connection context
0809 04:40:36.439384 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 04:40:36.439703 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":117,"threads_started":1}
I20260809 04:40:36.441627 13616 heartbeater.cc:347] Connected to a master server at 127.5.222.254:43161
I20260809 04:40:36.441883 13616 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:36.442313 13616 heartbeater.cc:511] Master 127.5.222.254:43161 requested a full tablet report, sending...
I20260809 04:40:36.443378 13080 ts_manager.cc:194] Registered new tserver with Master: 1a56bf57806d4996bf4de69813ea10d6 (127.5.222.195:38385)
I20260809 04:40:36.444099 13080 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:39901
I20260809 04:40:36.444200 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.222.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.253.keytab.
Entry for principal oryx/127.5.222.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.253.keytab.
I20260809 04:40:36.472487 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.253:44613
--webserver_interface=127.5.222.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.253.keytab
--principal=oryx/127.5.222.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:43161,127.5.222.253:44613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:36.591374 13623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:36.591655 13623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:36.591741 13623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:36.596258 13623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:36.596318 13623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:36.596338 13623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:36.596354 13623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:36.596406 13623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:36.602062 13623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:43161,127.5.222.253:44613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.253.keytab
--principal=oryx/127.5.222.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.253:44613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.222.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
W20260809 04:40:36.602265 13623 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.222.254:43161,127.5.222.253:44613'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 04:40:36.603119 13623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:36.604131 13623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:36.612556 13634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:36.612669 13632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:36.612653 13631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:36.612824 13623 server_base.cc:1034] running on GCE node
I20260809 04:40:36.613405 13623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:36.614512 13623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:36.615700 13623 hybrid_clock.cc:648] HybridClock initialized: now 1786250436615686 us; error 29 us; skew 500 ppm
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:36.618597 13623 init.cc:377] Logged in from keytab as oryx/127.5.222.253@KRBTEST.COM (short username oryx)
I20260809 04:40:36.619541 13623 webserver.cc:460] Webserver started at http://127.5.222.253:33695/ using document root <none> and password file <none>
I20260809 04:40:36.620112 13623 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:36.620205 13623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:36.620438 13623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:36.622540 13623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data/instance:
uuid: "7d405a7019604bb88d8db6f3656676c4"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "926beeba5005acc17c25ef9191f91022"
server_key_iv: "2a77f71581c5a250c20ec3944659a06f"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.623230 13623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/wal/instance:
uuid: "7d405a7019604bb88d8db6f3656676c4"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "926beeba5005acc17c25ef9191f91022"
server_key_iv: "2a77f71581c5a250c20ec3944659a06f"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.627246 13623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:40:36.629617 13642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:36.630772 13623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:36.630923 13623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/wal
uuid: "7d405a7019604bb88d8db6f3656676c4"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "926beeba5005acc17c25ef9191f91022"
server_key_iv: "2a77f71581c5a250c20ec3944659a06f"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.631031 13623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:36.649489 13623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:36.652779 13623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:36.652997 13623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:36.660106 13623 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.253:44613
I20260809 04:40:36.660423 13703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.253:44613 every 8 connection(s)
I20260809 04:40:36.661064 13623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
I20260809 04:40:36.664067 13704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:36.667754 13704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:40:36.668728 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 13623
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.253@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.222.253@KRBTEST.COM
I20260809 04:40:36.682776 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.669888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55292 (local address 127.5.222.254:43161)
0809 04:40:36.670018 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:36.670023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:36.670770 (+ 747us) server_negotiation.cc:407] Connection header received
0809 04:40:36.671559 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.671562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.671625 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.671711 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.672953 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.673543 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.674738 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.675025 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.675104 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.678645 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.678649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.678652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.678683 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:36.680461 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.681029 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.681033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.681035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.681126 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.681533 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.681536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.681538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.681812 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:36.681915 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:36.682485 (+ 570us) server_negotiation.cc:299] Negotiation successful
0809 04:40:36.682610 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:40:36.683830 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.670170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55292)
0809 04:40:36.670612 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:36.670659 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:36.671329 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:36.671833 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:36.671840 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:36.672389 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:36.672800 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.672823 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.673697 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.673701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.674609 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.674618 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.675184 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.675188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.675369 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.676030 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:36.676046 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:36.678455 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:36.680589 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.680598 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.680607 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.680919 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.681331 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.681335 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.681340 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.681451 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.681920 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:36.681924 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:36.682208 (+ 284us) client_negotiation.cc:769] Sending connection context
0809 04:40:36.682419 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 04:40:36.682749 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":143,"threads_started":1}
I20260809 04:40:36.683962 13708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.670171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.5.222.253:44613)
0809 04:40:36.670577 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:36.670592 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:36.670623 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:40:36.671295 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.671312 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.671536 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.671721 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.672868 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.673996 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.674905 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.675395 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.675504 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.678599 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.678605 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.678618 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.678644 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:36.680766 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.681291 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.681298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.681299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.681444 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.682175 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.682178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.682180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.682463 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:36.683491 (+ 1028us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:36.683618 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 04:40:36.683750 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"spinlock_wait_cycles":15232,"thread_start_us":169,"threads_started":1}
I20260809 04:40:36.685289 13704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.253@KRBTEST.COM for oryx/127.5.222.253@KRBTEST.COM
I20260809 04:40:36.693976 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.685778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:44613 (local address 127.0.0.1:53852)
0809 04:40:36.685885 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:36.685902 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:36.685972 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:36.686459 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:36.686462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:36.686626 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:36.686717 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.686722 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.687712 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.687714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.688630 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.688638 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.689261 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.689264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.689337 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.689825 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:36.689838 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:36.691080 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:36.692680 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.692683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.692685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.692866 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.693123 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.693125 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.693126 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.693256 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.693592 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:36.693594 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:36.693667 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 04:40:36.693745 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 04:40:36.693845 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 04:40:36.694123 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.685920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.5.222.253:44613)
0809 04:40:36.686221 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:36.686229 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:36.686240 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:40:36.686270 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.686272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.686309 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.686391 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.686821 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.687612 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.688747 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.689105 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.689249 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.691219 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.691222 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.691224 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.691246 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:36.692580 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.692958 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.692961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.692962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.693041 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.693335 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.693337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.693338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.693510 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:36.693589 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:36.693806 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 04:40:36.693920 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":118,"threads_started":1}
I20260809 04:40:36.696759 13704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: Bootstrap starting.
I20260809 04:40:36.699103 13704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:36.699891 13704 log.cc:824] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:36.702131 13704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: No bootstrap required, opened a new log
I20260809 04:40:36.705025 13704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } }
I20260809 04:40:36.705312 13704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:36.705376 13704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d405a7019604bb88d8db6f3656676c4, State: Initialized, Role: FOLLOWER
I20260809 04:40:36.705806 13704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } }
I20260809 04:40:36.706207 13711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } } }
I20260809 04:40:36.706343 13711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:40:36.706951 13704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:40:36.711354 13723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:40:36.711444 13723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:40:36.714022 13623 master_runner.cc:428] Detected that this master 7d405a7019604bb88d8db6f3656676c4 is joining an existing cluster
I20260809 04:40:36.714092 13623 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.222.253:44613
I20260809 04:40:36.724927 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.716485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55300 (local address 127.5.222.254:43161)
0809 04:40:36.716616 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:36.716620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:36.716910 (+ 290us) server_negotiation.cc:407] Connection header received
0809 04:40:36.717069 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.717071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.717114 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.717224 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.717882 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.718714 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.719694 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.719919 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.720030 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.721328 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.721331 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.721336 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.721363 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:36.723205 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.723692 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.723696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.723698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.723787 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.724083 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.724088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.724090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.724273 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:36.724382 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:36.724655 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 04:40:36.724774 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:40:36.724941 13729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.716398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55300)
0809 04:40:36.716807 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:36.716822 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:36.716945 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:36.717355 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:36.717358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:36.717657 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:36.717760 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.717765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.718855 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.718858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.719567 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.719576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.720042 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.720045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.720134 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.720682 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:36.720696 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:36.721144 (+ 448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:36.723331 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.723334 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.723337 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.723582 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.723881 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.723885 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.723887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.723987 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.724370 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:36.724373 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:36.724479 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 04:40:36.724600 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 04:40:36.724739 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":64,"spinlock_wait_cycles":896,"thread_start_us":107,"threads_started":1}
I20260809 04:40:36.725618 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.716398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:44613 (local address 127.0.0.1:53864)
0809 04:40:36.716944 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:36.716959 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:36.717080 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:36.717366 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:36.717369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:36.717700 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:36.717838 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.717857 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.718836 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.718853 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.720212 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.720224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.721137 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.721141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.721299 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.721747 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:36.721762 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:36.722175 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:36.724150 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.724152 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.724154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.724378 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.724631 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.724634 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.724636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.724704 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.725093 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:36.725095 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:36.725195 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 04:40:36.725313 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 04:40:36.725435 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":272,"thread_start_us":99,"threads_started":1}
I20260809 04:40:36.725656 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.716537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53864 (local address 127.5.222.253:44613)
0809 04:40:36.716636 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:36.716641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:36.717027 (+ 386us) server_negotiation.cc:407] Connection header received
0809 04:40:36.717198 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.717203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.717247 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.717320 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.717944 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.718711 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.720670 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.721011 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.721121 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.722317 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.722323 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.722329 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.722357 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:36.724036 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.724473 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.724476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.724478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.724544 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.724784 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.724787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.724789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.724974 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:36.725038 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:36.725376 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 04:40:36.725514 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":39}
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250435, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.254@KRBTEST.COM for oryx/127.5.222.253@KRBTEST.COM
I20260809 04:40:36.739638 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.729920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:44613 (local address 127.0.0.1:53878)
0809 04:40:36.730257 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:36.730307 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:36.730440 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:36.730740 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:36.730745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:36.731077 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:36.731181 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.731186 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.732137 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.732139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.732746 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.732754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.733311 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.733314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.733396 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.733905 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:36.733947 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:36.735556 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:36.737635 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.737645 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.737650 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.737972 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.738264 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.738270 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.738275 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.738379 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.738806 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:36.738815 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:36.738977 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 04:40:36.739143 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 04:40:36.739377 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":106,"threads_started":1}
I20260809 04:40:36.739715 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.729943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53878 (local address 127.5.222.253:44613)
0809 04:40:36.730047 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:36.730051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:36.730390 (+ 339us) server_negotiation.cc:407] Connection header received
0809 04:40:36.730568 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.730572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.730620 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.730704 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.731286 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.732031 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.732859 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.733211 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.733309 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.735682 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.735686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.735689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.735713 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:36.737521 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.738074 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.738078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.738080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.738162 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.738480 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.738484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.738485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.738689 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:36.738791 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:36.739208 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 04:40:36.739549 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:40:36.741665 13080 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.222.253:44613: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7d405a7019604bb88d8db6f3656676c4"
member_type: NON_VOTER
last_known_addr {
host: "127.5.222.253"
port: 44613
}
attrs {
promote: true
}
}
dest_uuid: "086656952f8644e4a572db00a1becc1f"
cas_config_opid_index: -1
I20260809 04:40:36.742333 13080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true } }
I20260809 04:40:36.743852 13734 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7d405a7019604bb88d8db6f3656676c4 (127.5.222.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true } } }
I20260809 04:40:36.744829 13673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:36.745343 13734 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.222.253:44613
I20260809 04:40:36.745590 13734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true } } }
I20260809 04:40:36.745687 13734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:36.745846 13623 master.cc:537] Master@127.5.222.253:44613 shutting down...
I20260809 04:40:36.746521 13673 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 FOLLOWER]: Refusing update from remote peer 086656952f8644e4a572db00a1becc1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
WARNING: no policy specified for oryx/127.5.222.252@KRBTEST.COM; defaulting to no policy
I20260809 04:40:36.747395 13733 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.252@KRBTEST.COM" created.
W20260809 04:40:36.749818 13065 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 04:40:36.749979 13623 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:36.750253 13623 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:36.750373 13623 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: stopping tablet replica
W20260809 04:40:36.750660 13065 consensus_peers.cc:487] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f -> Peer 7d405a7019604bb88d8db6f3656676c4 (127.5.222.253:44613): Couldn't send request to peer 7d405a7019604bb88d8db6f3656676c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 04:40:36.764833 13623 master.cc:552] Master@127.5.222.253:44613 shutdown complete.
I20260809 04:40:36.764914 13623 master_runner.cc:305] Clearing existing system tablet
I20260809 04:40:36.765244 13623 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:36.765312 13623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:36.766558 13623 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 04:40:36.767480 13741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.252.keytab.
Entry for principal oryx/127.5.222.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.252.keytab.
I20260809 04:40:36.767750 13623 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:36.767877 13623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 04:40:36.767931 13623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/wal
uuid: "7d405a7019604bb88d8db6f3656676c4"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "926beeba5005acc17c25ef9191f91022"
server_key_iv: "2a77f71581c5a250c20ec3944659a06f"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.768684 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.252:35257
--webserver_interface=127.5.222.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.252.keytab
--principal=oryx/127.5.222.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:43161,127.5.222.253:44613,127.5.222.252:35257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:40:36.769841 13623 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:36.773089 13623 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 04:40:36.773209 13623 log.cc:1196] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 04:40:36.773490 13623 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: Deleting consensus metadata
I20260809 04:40:36.773728 13623 master_runner.cc:315] Copying system tablet from 127.5.222.254:43161
I20260809 04:40:36.775126 13623 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: tablet copy: Beginning tablet copy session from remote peer at address 127.5.222.254:43161
I20260809 04:40:36.784749 13749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.775725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55316)
0809 04:40:36.776045 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:36.776065 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:36.776199 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:36.776489 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:36.776494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:36.776728 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:36.776902 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.776927 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.778119 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.778126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.779205 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.779221 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.779831 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.779838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.779980 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.780600 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:36.780618 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:36.781133 (+ 515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:36.783161 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.783165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.783167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.783419 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.783710 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.783713 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.783715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.783806 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.784209 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:36.784212 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:36.784309 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 04:40:36.784415 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 04:40:36.784557 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":92,"threads_started":1}
I20260809 04:40:36.784766 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.776110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55316 (local address 127.5.222.254:43161)
0809 04:40:36.776227 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:36.776231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:36.776250 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:40:36.776298 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.776301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.776350 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.776439 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.777063 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.777974 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.779342 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.779692 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.779805 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.781313 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.781317 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.781321 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.781349 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:36.783052 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.783523 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.783527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.783529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.783615 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.783901 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.783904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.783906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.784114 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:36.784209 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:36.784490 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 04:40:36.784610 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:40:36.785473 13102 tablet_copy_service.cc:140] P 086656952f8644e4a572db00a1becc1f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7d405a7019604bb88d8db6f3656676c4 ({username='oryx', principal='oryx/127.5.222.253@KRBTEST.COM'} at 127.0.0.1:55316)
I20260809 04:40:36.785679 13102 tablet_copy_service.cc:161] P 086656952f8644e4a572db00a1becc1f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7d405a7019604bb88d8db6f3656676c4 at {username='oryx', principal='oryx/127.5.222.253@KRBTEST.COM'} at 127.0.0.1:55316: session id = 7d405a7019604bb88d8db6f3656676c4-00000000000000000000000000000000
I20260809 04:40:36.787454 13102 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 04:40:36.789170 13623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 04:40:36.791432 13623 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: tablet copy: Starting download of 0 data blocks...
I20260809 04:40:36.791617 13623 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: tablet copy: Starting download of 1 WAL segments...
I20260809 04:40:36.793329 13623 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 04:40:36.795042 13102 tablet_copy_service.cc:342] P 086656952f8644e4a572db00a1becc1f: Request end of tablet copy session 7d405a7019604bb88d8db6f3656676c4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.222.253@KRBTEST.COM'} at 127.0.0.1:55316
I20260809 04:40:36.795156 13102 tablet_copy_service.cc:433] P 086656952f8644e4a572db00a1becc1f: ending tablet copy session 7d405a7019604bb88d8db6f3656676c4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7d405a7019604bb88d8db6f3656676c4
I20260809 04:40:36.802528 13623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:36.804255 13750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:36.804252 13751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:36.805004 13623 server_base.cc:1034] running on GCE node
W20260809 04:40:36.808993 13753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:36.809306 13623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:36.809607 13623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:36.810750 13623 hybrid_clock.cc:648] HybridClock initialized: now 1786250436810729 us; error 43 us; skew 500 ppm
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:36.812340 13623 init.cc:377] Logged in from keytab as oryx/127.5.222.253@KRBTEST.COM (short username oryx)
I20260809 04:40:36.812824 13623 webserver.cc:460] Webserver started at http://127.5.222.253:44809/ using document root <none> and password file <none>
I20260809 04:40:36.813019 13623 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:36.813064 13623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:36.815778 13623 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:40:36.817492 13761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:36.817775 13623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 04:40:36.817850 13623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/wal
uuid: "7d405a7019604bb88d8db6f3656676c4"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "926beeba5005acc17c25ef9191f91022"
server_key_iv: "2a77f71581c5a250c20ec3944659a06f"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.817943 13623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:36.830264 13623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:36.832410 13623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:36.837227 13623 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.253:44613
I20260809 04:40:36.837262 13820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.253:44613 every 8 connection(s)
I20260809 04:40:36.837793 13623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
I20260809 04:40:36.839783 13821 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:40:36.840813 13821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: Bootstrap starting.
I20260809 04:40:36.851227 13821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:40:36.851542 13821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: Bootstrap complete.
I20260809 04:40:36.852109 13821 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true } }
I20260809 04:40:36.852248 13821 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7d405a7019604bb88d8db6f3656676c4, State: Initialized, Role: LEARNER
I20260809 04:40:36.852365 13821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true } }
I20260809 04:40:36.852720 13824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true } } }
I20260809 04:40:36.852824 13821 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:36.852929 13824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:40:36.856738 13837 catalog_manager.cc:1259] Loaded cluster ID: c2bfbc2767904c37bd7cfea1ce85703e
I20260809 04:40:36.856797 13837 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: loading cluster ID for follower catalog manager: success
I20260809 04:40:36.859018 13837 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: acquiring CA information for follower catalog manager: success
I20260809 04:40:36.860316 13837 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 04:40:36.899258 13742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:36.899509 13742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:36.899550 13742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:36.903906 13742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:36.903964 13742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:36.903985 13742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:36.904000 13742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:36.904016 13742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:36.909240 13742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:43161,127.5.222.253:44613,127.5.222.252:35257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.252.keytab
--principal=oryx/127.5.222.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.252:35257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.222.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:36.910236 13742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:36.911257 13742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:36.919749 13847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:36.919752 13843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:36.919838 13742 server_base.cc:1034] running on GCE node
W20260809 04:40:36.919725 13845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:36.920621 13742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:36.921684 13742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:36.922845 13742 hybrid_clock.cc:648] HybridClock initialized: now 1786250436922830 us; error 26 us; skew 500 ppm
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:36.925647 13742 init.cc:377] Logged in from keytab as oryx/127.5.222.252@KRBTEST.COM (short username oryx)
I20260809 04:40:36.926497 13742 webserver.cc:460] Webserver started at http://127.5.222.252:42949/ using document root <none> and password file <none>
I20260809 04:40:36.927039 13742 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:36.927129 13742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:36.927362 13742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:36.929471 13742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data/instance:
uuid: "c39ed4367f384162a78b1aafad4cde45"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "981e50bd2db76478bf61fca40e89abaf"
server_key_iv: "426c3adae98f492f5eb23dbd58cec518"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.930083 13742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/wal/instance:
uuid: "c39ed4367f384162a78b1aafad4cde45"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "981e50bd2db76478bf61fca40e89abaf"
server_key_iv: "426c3adae98f492f5eb23dbd58cec518"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.933642 13742 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:40:36.935937 13854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:36.937021 13742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:36.937203 13742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/wal
uuid: "c39ed4367f384162a78b1aafad4cde45"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "981e50bd2db76478bf61fca40e89abaf"
server_key_iv: "426c3adae98f492f5eb23dbd58cec518"
server_key_version: "encryptionkey@0"
I20260809 04:40:36.937314 13742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:36.961584 13742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:36.964768 13742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:36.964960 13742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:36.971626 13742 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.252:35257
I20260809 04:40:36.971909 13909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.252:35257 every 8 connection(s)
I20260809 04:40:36.972623 13742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
I20260809 04:40:36.975455 13913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:36.976078 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 13742
I20260809 04:40:36.981351 13913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.222.252@KRBTEST.COM
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.252@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:36.991613 13915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.977489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40502 (local address 127.5.222.252:35257)
0809 04:40:36.977929 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:36.977944 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:36.977978 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:40:36.978944 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.978966 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.979365 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.979845 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.980481 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.981964 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.982804 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.983353 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.983527 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.986598 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.986605 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.986618 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.986655 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:36.988353 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.988790 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.988803 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.988826 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.988974 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.989288 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.989293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.989294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.989821 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:36.990056 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:36.990290 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 04:40:36.990606 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":113,"threads_started":1}
I20260809 04:40:36.992882 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.983071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55324)
0809 04:40:36.983411 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:36.983457 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:36.983612 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:36.983985 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:36.983991 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:36.984399 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:36.984533 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.984539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.985660 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.985664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.986560 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.986572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.987171 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.987174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.987256 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.987840 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:36.987854 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:36.989204 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:36.991270 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.991278 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.991282 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.991492 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.991817 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:36.991820 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:36.991822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:36.991928 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:36.992300 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:36.992305 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:36.992411 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 04:40:36.992554 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 04:40:36.992724 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":122,"threads_started":1}
I20260809 04:40:36.992919 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.983241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.5.222.254:43161)
0809 04:40:36.983387 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:36.983393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:36.983529 (+ 136us) server_negotiation.cc:407] Connection header received
0809 04:40:36.983762 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.983769 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.983842 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.983912 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.984665 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.985504 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.986680 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.987017 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.987157 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.989370 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.989374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.989377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.989406 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:36.991161 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.991593 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.991599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.991604 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.991701 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:36.992027 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:36.992030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:36.992031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:36.992199 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:36.992277 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:36.992637 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 04:40:36.992752 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:40:36.993964 13913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:40:36 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.252@KRBTEST.COM for oryx/127.5.222.253@KRBTEST.COM
I20260809 04:40:37.003330 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.994565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53880 (local address 127.5.222.253:44613)
0809 04:40:36.994826 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:36.994829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:36.994841 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:40:36.994884 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:36.994886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:36.994923 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:36.994997 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:36.995418 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.996383 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.997108 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.997496 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.997612 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.999619 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:36.999622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:36.999624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:36.999648 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.001541 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.002093 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.002096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.002098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.002204 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.002501 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.002505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.002507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.002688 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.002801 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.003052 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.003171 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":109,"threads_started":1}
I20260809 04:40:37.003326 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:36.994436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:44613 (local address 127.0.0.1:53880)
0809 04:40:36.994556 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:36.994571 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:36.994645 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:36.995026 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:36.995028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:36.995149 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:36.995273 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.995282 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.996485 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.996488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.996994 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:36.997000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:36.997604 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:36.997606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:36.997679 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:36.998158 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:36.998169 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:36.999483 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.001687 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.001693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.001696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.001995 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.002313 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.002316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.002318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.002408 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.002783 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.002786 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.002876 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.002990 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.003154 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 04:40:37.004434 13913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.252@KRBTEST.COM for oryx/127.5.222.252@KRBTEST.COM
I20260809 04:40:37.011688 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.004906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:35257 (local address 127.0.0.1:40506)
0809 04:40:37.005018 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.005033 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.005107 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.005340 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.005342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.005463 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.005530 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.005534 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.006571 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.006574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.007055 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.007062 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.007480 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.007483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.007539 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.007894 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.007904 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.009023 (+ 1119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.010535 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.010538 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.010540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.010723 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.010945 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.010947 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.010949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.011016 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.011321 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.011323 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.011389 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.011465 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.011551 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 04:40:37.011724 13915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.004982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.5.222.252:35257)
0809 04:40:37.005059 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.005062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.005073 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:40:37.005223 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.005225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.005257 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.005318 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.005619 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.006111 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.007151 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.007386 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.007460 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.009148 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.009187 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.009189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.009219 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.010437 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.010801 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.010803 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.010804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.010872 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.011086 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.011088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.011089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.011234 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.011303 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.011517 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.011604 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 04:40:37.014748 13913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: Bootstrap starting.
I20260809 04:40:37.017024 13913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:37.017853 13913 log.cc:824] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:37.019887 13913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: No bootstrap required, opened a new log
I20260809 04:40:37.021708 13300 heartbeater.cc:503] Master 127.5.222.254:43161 was elected leader, sending a full tablet report...
I20260809 04:40:37.023248 13913 raft_consensus.cc:348] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } }
I20260809 04:40:37.023494 13913 raft_consensus.cc:374] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:37.023576 13913 raft_consensus.cc:729] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c39ed4367f384162a78b1aafad4cde45, State: Initialized, Role: FOLLOWER
I20260809 04:40:37.024041 13913 consensus_queue.cc:260] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } }
I20260809 04:40:37.024508 13920 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } } }
I20260809 04:40:37.024622 13920 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:40:37.025218 13913 sys_catalog.cc:564] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:40:37.030170 13931 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:40:37.030261 13931 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:40:37.033620 13742 master_runner.cc:428] Detected that this master c39ed4367f384162a78b1aafad4cde45 is joining an existing cluster
I20260809 04:40:37.033699 13742 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.222.252:35257
I20260809 04:40:37.045127 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 13152
I20260809 04:40:37.059696 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:38087
--local_ip_for_outbound_sockets=127.5.222.193
--tserver_master_addrs=127.5.222.254:43161
--webserver_port=42013
--webserver_interface=127.5.222.193
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab
--principal=oryx/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.222.254:43161,127.5.222.253:44613
--tserver_master_addrs=127.5.222.254:43161,127.5.222.253:44613,127.5.222.252:35257 with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:40:37.065622 13915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.036722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40510 (local address 127.5.222.252:35257)
0809 04:40:37.036861 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.036865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.037660 (+ 795us) server_negotiation.cc:407] Connection header received
0809 04:40:37.037813 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.037816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.037848 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.037893 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.038574 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.039214 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.040815 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.041188 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.041257 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.062050 (+ 20793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.062054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.062057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.062081 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.063909 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.064338 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.064342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.064343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.064409 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.064637 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.064639 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.064641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.064911 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.064980 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.065326 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.065456 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 04:40:37.066363 13943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.036686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:35257 (local address 127.0.0.1:40510)
0809 04:40:37.037457 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.037471 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.037723 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.038002 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.038008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.038222 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.038453 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.038460 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.039330 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.039332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.040705 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.040716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.041385 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.041390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.041496 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.041958 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.041973 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.061892 (+ 19919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.064033 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.064036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.064038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.064250 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.064492 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.064495 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.064497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.064561 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.065039 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.065042 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.065169 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.066046 (+ 877us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.066183 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":49,"thread_start_us":284,"threads_started":1}
I20260809 04:40:37.066749 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.036203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55326)
0809 04:40:37.036559 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.036576 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.037001 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.037433 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.037436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.037763 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.037950 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.037962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.039020 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.039023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.040005 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.040014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.040822 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.040825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.040920 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.041497 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.041511 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.061904 (+ 20393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.064287 (+ 2383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.064293 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.064298 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.064554 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.064936 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.064940 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.064945 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.065037 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.065867 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.065872 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.066242 (+ 370us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.066363 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.066530 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":24,"thread_start_us":150,"threads_started":1}
I20260809 04:40:37.067078 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.036169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.5.222.254:43161)
0809 04:40:37.036389 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.036393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.036805 (+ 412us) server_negotiation.cc:407] Connection header received
0809 04:40:37.037135 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.037140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.037218 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.037278 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.038086 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.038900 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.040143 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.040545 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.040646 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.062111 (+ 21465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.062115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.062118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.062152 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.064158 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.064678 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.064685 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.064689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.064782 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.065143 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.065184 (+ 41us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.065186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.065703 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.065805 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.066805 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.066927 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260809 04:40:37.069072 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.036368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53890 (local address 127.5.222.253:44613)
0809 04:40:37.036486 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.036491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.037474 (+ 983us) server_negotiation.cc:407] Connection header received
0809 04:40:37.037646 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.037649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.037692 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.037742 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.038558 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.039432 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.040731 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.041109 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.041649 (+ 540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.065042 (+ 23393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.065047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.065049 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.065081 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.066869 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.067427 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.067432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.067436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.067532 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.067887 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.067894 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.067899 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.068153 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.068236 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.068662 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.068825 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:40:37.069844 13942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.036203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:44613 (local address 127.0.0.1:53890)
0809 04:40:37.037253 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.037267 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.037539 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.037897 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.037900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.038195 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.038389 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.038398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.039570 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.039573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.040594 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.040604 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.041811 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.041815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.041923 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.042328 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.042343 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.063234 (+ 20891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.066999 (+ 3765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.067004 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.067009 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.067284 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.067665 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.067670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.067673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.067771 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.068321 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.068327 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.068447 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.069404 (+ 957us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.069580 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":66,"thread_start_us":103,"threads_started":1}
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250435, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.254@KRBTEST.COM for oryx/127.5.222.252@KRBTEST.COM
I20260809 04:40:37.081944 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.072558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:35257 (local address 127.0.0.1:40516)
0809 04:40:37.072718 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.072732 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.072812 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.073191 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.073195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.073380 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.073481 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.073487 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.074597 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.074600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.075482 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.075492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.076105 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.076109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.076205 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.076716 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.076730 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.078427 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.080384 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.080387 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.080389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.080635 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.080926 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.080929 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.080931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.081019 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.081437 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.081440 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.081534 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.081639 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.081774 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 04:40:37.081974 13915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.072646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40516 (local address 127.5.222.252:35257)
0809 04:40:37.072772 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.072777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.072792 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:40:37.072924 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.072927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.073024 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.073144 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.073608 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.074467 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.075618 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.075978 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.076089 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.078563 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.078566 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.078569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.078594 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.080259 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.080734 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.080737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.080739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.080828 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.081113 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.081116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.081118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.081338 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.081433 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.081709 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.081827 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:40:37.083302 13080 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.222.252:35257: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c39ed4367f384162a78b1aafad4cde45"
member_type: NON_VOTER
last_known_addr {
host: "127.5.222.252"
port: 35257
}
attrs {
promote: true
}
}
dest_uuid: "086656952f8644e4a572db00a1becc1f"
cas_config_opid_index: 5
I20260809 04:40:37.083918 13080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } }
I20260809 04:40:37.085757 13882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:37.087294 13882 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 1 FOLLOWER]: Refusing update from remote peer 086656952f8644e4a572db00a1becc1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 04:40:37.087875 13734 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c39ed4367f384162a78b1aafad4cde45 (127.5.222.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.089429 13734 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.222.252:35257
I20260809 04:40:37.089605 13734 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:37.089917 13742 master.cc:537] Master@127.5.222.252:35257 shutting down...
I20260809 04:40:37.090360 13734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.090479 13734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: This master's current role is: LEADER
W20260809 04:40:37.092593 13065 consensus_peers.cc:487] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f -> Peer c39ed4367f384162a78b1aafad4cde45 (127.5.222.252:35257): Couldn't send request to peer c39ed4367f384162a78b1aafad4cde45. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 04:40:37.094973 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.085194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:44613 (local address 127.0.0.1:53896)
0809 04:40:37.085301 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.085316 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.085391 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.091070 (+ 5679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.091073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.091090 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:37.091168 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.091173 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.092177 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.092180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.093411 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.093422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.094556 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.094560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.094661 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.094722 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.094793 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.094858 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 04:40:37.095359 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.087759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53896 (local address 127.5.222.253:44613)
0809 04:40:37.090732 (+ 2973us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.090736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.090751 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:40:37.090800 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.090803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.090856 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.090949 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:37.091284 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.092041 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.093550 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.094147 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.094248 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.094394 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.095162 (+ 768us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.095236 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2885}
I20260809 04:40:37.095764 13792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 LEARNER]: Refusing update from remote peer 086656952f8644e4a572db00a1becc1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 04:40:37.096149 13742 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:37.096221 13945 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 04:40:37.096417 13742 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:37.096530 13742 tablet_replica.cc:330] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: stopping tablet replica
I20260809 04:40:37.098069 13792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c39ed4367f384162a78b1aafad4cde45 (127.5.222.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.100430 13947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.100545 13947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:40:37.100510 13824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 086656952f8644e4a572db00a1becc1f. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: NON_VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: true } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.100648 13824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:40:37.112598 13733 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f: attempting to promote NON_VOTER 7d405a7019604bb88d8db6f3656676c4 to VOTER
I20260809 04:40:37.113134 13733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } }
I20260809 04:40:37.113392 13742 master.cc:552] Master@127.5.222.252:35257 shutdown complete.
I20260809 04:40:37.113443 13742 master_runner.cc:305] Clearing existing system tablet
I20260809 04:40:37.113693 13742 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:37.113744 13742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:37.115108 13792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 LEARNER]: Refusing update from remote peer 086656952f8644e4a572db00a1becc1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 04:40:37.115376 13742 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:37.115974 13734 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 04:40:37.116343 13949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:37.116552 13742 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:37.116657 13742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:37.116694 13742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/wal
uuid: "c39ed4367f384162a78b1aafad4cde45"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "981e50bd2db76478bf61fca40e89abaf"
server_key_iv: "426c3adae98f492f5eb23dbd58cec518"
server_key_version: "encryptionkey@0"
I20260809 04:40:37.116773 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.114615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:35257 (local address 127.0.0.1:40530)
0809 04:40:37.116595 (+ 1980us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.116685 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:35257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1896}
I20260809 04:40:37.117645 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.117324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:35257 (local address 127.0.0.1:40536)
0809 04:40:37.117496 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.117543 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:35257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260809 04:40:37.117756 13065 consensus_peers.cc:487] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f -> Peer c39ed4367f384162a78b1aafad4cde45 (127.5.222.252:35257): Couldn't send request to peer c39ed4367f384162a78b1aafad4cde45. Status: Network error: Client connection negotiation failed: client connection to 127.5.222.252:35257: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 04:40:37.118712 13742 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:37.118744 13734 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7d405a7019604bb88d8db6f3656676c4 (127.5.222.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.120335 13734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.120437 13734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:37.121094 13733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.121248 13742 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 04:40:37.121264 13733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:37.121359 13742 log.cc:1196] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 04:40:37.121620 13742 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: Deleting consensus metadata
I20260809 04:40:37.121788 13742 master_runner.cc:315] Copying system tablet from 127.5.222.254:43161
I20260809 04:40:37.122236 13791 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7d405a7019604bb88d8db6f3656676c4 (127.5.222.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.122905 13742 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: tablet copy: Beginning tablet copy session from remote peer at address 127.5.222.254:43161
I20260809 04:40:37.124274 13824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.124404 13824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:40:37.132975 13957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.123769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55340)
0809 04:40:37.124041 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.124055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.124211 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.124561 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.124564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.124770 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.124883 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.124891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.125990 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.125994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.127048 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.127059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.127616 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.127620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.127728 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.128716 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.128730 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.129140 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.131250 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.131254 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.131256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.131511 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.131837 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.131840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.131842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.131967 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.132398 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.132401 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.132498 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.132614 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.132768 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":92,"threads_started":1}
I20260809 04:40:37.132977 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.123579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55340 (local address 127.5.222.254:43161)
0809 04:40:37.123681 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.123686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.124139 (+ 453us) server_negotiation.cc:407] Connection header received
0809 04:40:37.124327 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.124331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.124376 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.124427 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.125021 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.125855 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.127162 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.127495 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.127601 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.129335 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.129339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.129342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.129369 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.131055 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.131623 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.131628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.131630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.131731 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.132063 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.132066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.132068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.132287 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.132398 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.132712 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.132832 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 04:40:37.133792 13102 tablet_copy_service.cc:140] P 086656952f8644e4a572db00a1becc1f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c39ed4367f384162a78b1aafad4cde45 ({username='oryx', principal='oryx/127.5.222.252@KRBTEST.COM'} at 127.0.0.1:55340)
I20260809 04:40:37.133916 13102 tablet_copy_service.cc:161] P 086656952f8644e4a572db00a1becc1f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c39ed4367f384162a78b1aafad4cde45 at {username='oryx', principal='oryx/127.5.222.252@KRBTEST.COM'} at 127.0.0.1:55340: session id = c39ed4367f384162a78b1aafad4cde45-00000000000000000000000000000000
I20260809 04:40:37.135123 13102 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 04:40:37.136396 13742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 04:40:37.138710 13742 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: tablet copy: Starting download of 0 data blocks...
I20260809 04:40:37.138902 13742 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: tablet copy: Starting download of 1 WAL segments...
I20260809 04:40:37.140446 13742 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 04:40:37.142159 13102 tablet_copy_service.cc:342] P 086656952f8644e4a572db00a1becc1f: Request end of tablet copy session c39ed4367f384162a78b1aafad4cde45-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.222.252@KRBTEST.COM'} at 127.0.0.1:55340
I20260809 04:40:37.142241 13102 tablet_copy_service.cc:433] P 086656952f8644e4a572db00a1becc1f: ending tablet copy session c39ed4367f384162a78b1aafad4cde45-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c39ed4367f384162a78b1aafad4cde45
I20260809 04:40:37.149442 13742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:37.151319 13958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:37.151647 13959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:37.151846 13961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:37.153700 13742 server_base.cc:1034] running on GCE node
I20260809 04:40:37.153867 13742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:37.154147 13742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:37.155433 13742 hybrid_clock.cc:648] HybridClock initialized: now 1786250437155414 us; error 38 us; skew 500 ppm
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:37.157209 13742 init.cc:377] Logged in from keytab as oryx/127.5.222.252@KRBTEST.COM (short username oryx)
I20260809 04:40:37.157740 13742 webserver.cc:460] Webserver started at http://127.5.222.252:43461/ using document root <none> and password file <none>
I20260809 04:40:37.157935 13742 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:37.157986 13742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:37.159339 13742 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:37.160415 13969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:37.160696 13742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:37.160774 13742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/wal
uuid: "c39ed4367f384162a78b1aafad4cde45"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "981e50bd2db76478bf61fca40e89abaf"
server_key_iv: "426c3adae98f492f5eb23dbd58cec518"
server_key_version: "encryptionkey@0"
I20260809 04:40:37.160871 13742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:37.204578 13742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:37.206756 13742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:37.211874 13742 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.252:35257
I20260809 04:40:37.212594 13742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
I20260809 04:40:37.212787 14032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.252:35257 every 8 connection(s)
I20260809 04:40:37.214129 14033 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:40:37.214936 14033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: Bootstrap starting.
W20260809 04:40:37.216923 13944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:37.217141 13944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:37.217265 13944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 04:40:37.220151 13465 heartbeater.cc:503] Master 127.5.222.254:43161 was elected leader, sending a full tablet report...
W20260809 04:40:37.221993 13944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:37.222045 13944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:37.222137 13944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:37.223803 14033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:40:37.224073 14033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: Bootstrap complete.
I20260809 04:40:37.224530 14033 raft_consensus.cc:348] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } }
I20260809 04:40:37.224668 14033 raft_consensus.cc:729] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c39ed4367f384162a78b1aafad4cde45, State: Initialized, Role: LEARNER
I20260809 04:40:37.224797 14033 consensus_queue.cc:260] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } }
I20260809 04:40:37.225134 14036 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.225241 14033 sys_catalog.cc:564] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:37.225319 14036 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:40:37.228228 14048 catalog_manager.cc:1259] Loaded cluster ID: c2bfbc2767904c37bd7cfea1ce85703e
I20260809 04:40:37.228348 14048 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: loading cluster ID for follower catalog manager: success
I20260809 04:40:37.228325 13944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab
--principal=oryx/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:38087
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=42013
--tserver_master_addrs=127.5.222.254:43161,127.5.222.253:44613,127.5.222.252:35257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:37.229415 13944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:37.230015 14048 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: acquiring CA information for follower catalog manager: success
I20260809 04:40:37.230448 13944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:40:37.231092 14048 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 04:40:37.239895 14051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:37.239895 14050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:37.240147 13944 server_base.cc:1034] running on GCE node
W20260809 04:40:37.239914 14053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:37.240563 13944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:37.241282 13944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:37.242481 13944 hybrid_clock.cc:648] HybridClock initialized: now 1786250437242451 us; error 47 us; skew 500 ppm
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:37.245217 13944 init.cc:377] Logged in from keytab as oryx/127.5.222.193@KRBTEST.COM (short username oryx)
I20260809 04:40:37.246043 13944 webserver.cc:460] Webserver started at http://127.5.222.193:42013/ using document root <none> and password file <none>
I20260809 04:40:37.246622 13944 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:37.246739 13944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:37.250437 13944 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 04:40:37.252425 14061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:37.253743 13944 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:37.253875 13944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "8750546d7c214df3902dfe9a782667c0"
format_stamp: "Formatted at 2026-08-09 04:40:35 on dist-test-slave-vxj2"
server_key: "924ab39fc2d9a21dea35d579b76b0dcb"
server_key_iv: "af1e8439c95f2fec6282cc4b96f81600"
server_key_version: "encryptionkey@0"
I20260809 04:40:37.254400 13944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:37.270032 13944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:37.272935 13944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:37.273138 13944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:37.273885 13944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:37.274734 13944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:37.274804 13944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:37.274871 13944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:37.274931 13944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:37.285027 13944 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:38087
I20260809 04:40:37.285151 14188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:38087 every 8 connection(s)
I20260809 04:40:37.286202 13944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:37.293898 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 13944
I20260809 04:40:37.294121 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 13307
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.253@KRBTEST.COM
I20260809 04:40:37.303972 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.288363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:48973 (local address 127.5.222.253:44613)
0809 04:40:37.288510 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.288514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.289564 (+ 1050us) server_negotiation.cc:407] Connection header received
0809 04:40:37.290870 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.290873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.290923 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.291003 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.292708 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.293514 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.295317 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.295697 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.295806 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.299525 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.299531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.299536 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.299569 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.301568 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.302190 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.302195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.302200 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.302280 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.302689 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.302695 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.302699 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.302917 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.302987 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.303586 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.303709 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:40:37.305449 14194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.288643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:44613 (local address 127.5.222.193:48973)
0809 04:40:37.289273 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.289322 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.290595 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.292198 (+ 1603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.292201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.292376 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.292525 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.292538 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.293658 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.293661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.295178 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.295193 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.296051 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.296060 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.296361 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.297319 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.297350 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.299339 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.301743 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.301752 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.301767 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.302082 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.302382 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.302387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.302391 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.302557 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.303063 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.303071 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.303336 (+ 265us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.304114 (+ 778us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.304460 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":59,"thread_start_us":223,"threads_started":1}
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:37.310238 14192 heartbeater.cc:347] Connected to a master server at 127.5.222.253:44613
I20260809 04:40:37.310496 14192 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:37.311059 14192 heartbeater.cc:511] Master 127.5.222.253:44613 requested a full tablet report, sending...
I20260809 04:40:37.311568 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:43927
--local_ip_for_outbound_sockets=127.5.222.194
--tserver_master_addrs=127.5.222.254:43161
--webserver_port=46227
--webserver_interface=127.5.222.194
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab
--principal=oryx/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.222.254:43161,127.5.222.253:44613
--tserver_master_addrs=127.5.222.254:43161,127.5.222.253:44613,127.5.222.252:35257 with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:40:37.312467 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.289650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:55187 (local address 127.5.222.254:43161)
0809 04:40:37.289835 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.289842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.290260 (+ 418us) server_negotiation.cc:407] Connection header received
0809 04:40:37.290844 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.290847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.290902 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.291003 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.292669 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.293513 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.305064 (+ 11551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.305403 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.305490 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.307510 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.307517 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.307523 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.307558 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.309294 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.310273 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.310279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.310283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.310443 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.310827 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.310833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.310838 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.311086 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.311233 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.311601 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.312238 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:40:37.313102 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.289413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.193:55187)
0809 04:40:37.289889 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.289902 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.290597 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.291243 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.291251 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.291832 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.292522 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.292536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.304069 (+ 11533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.304073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.304920 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.304930 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.305652 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.305655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.305741 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.306206 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.306217 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.307367 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.309432 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.309439 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.309444 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.309714 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.310597 (+ 883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.310603 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.310608 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.310704 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.311306 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.311309 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.311411 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.312627 (+ 1216us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.312809 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":69,"spinlock_wait_cycles":17664,"thread_start_us":119,"threads_started":1}
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.252@KRBTEST.COM
I20260809 04:40:37.313890 14191 heartbeater.cc:347] Connected to a master server at 127.5.222.254:43161
I20260809 04:40:37.313975 14191 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:37.314147 14191 heartbeater.cc:511] Master 127.5.222.254:43161 requested a full tablet report, sending...
I20260809 04:40:37.314134 13782 ts_manager.cc:194] Registered new tserver with Master: 8750546d7c214df3902dfe9a782667c0 (127.5.222.193:38087)
I20260809 04:40:37.314986 13080 ts_manager.cc:194] Re-registered known tserver with Master: 8750546d7c214df3902dfe9a782667c0 (127.5.222.193:38087)
I20260809 04:40:37.315552 13080 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:55187
I20260809 04:40:37.320456 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.290119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:38511 (local address 127.5.222.252:35257)
0809 04:40:37.290388 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.290393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.291238 (+ 845us) server_negotiation.cc:407] Connection header received
0809 04:40:37.291349 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.291351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.291391 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.291583 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.292687 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.293489 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.310328 (+ 16839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.310690 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.310796 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.317092 (+ 6296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.317095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.317097 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.317123 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.318716 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.319282 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.319285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.319286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.319361 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.319673 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.319675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.319677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.319847 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.319931 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.320237 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.320336 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":71,"threads_started":1}
I20260809 04:40:37.320652 14198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.289746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:35257 (local address 127.5.222.193:38511)
0809 04:40:37.291042 (+ 1296us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.291055 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.291260 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.291639 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.291644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.291881 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.292527 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.292538 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.293629 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.293637 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.310154 (+ 16517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.310162 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.310966 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.310971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.311057 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.311517 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.311532 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.315992 (+ 4460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.318867 (+ 2875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.318872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.318876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.319179 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.319460 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.319465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.319469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.319576 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.319948 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.319954 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.320067 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.320193 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.320371 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1234,"mutex_wait_us":120,"spinlock_wait_cycles":1280,"thread_start_us":96,"threads_started":1}
I20260809 04:40:37.321486 14189 heartbeater.cc:347] Connected to a master server at 127.5.222.252:35257
I20260809 04:40:37.321584 14189 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:37.321827 14189 heartbeater.cc:511] Master 127.5.222.252:35257 requested a full tablet report, sending...
I20260809 04:40:37.322894 13988 ts_manager.cc:194] Registered new tserver with Master: 8750546d7c214df3902dfe9a782667c0 (127.5.222.193:38087)
W20260809 04:40:37.429207 14200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:37.429454 14200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:37.429530 14200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:37.433607 14200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:37.433661 14200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:37.433754 14200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:37.438943 14200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab
--principal=oryx/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:43927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=46227
--tserver_master_addrs=127.5.222.254:43161,127.5.222.253:44613,127.5.222.252:35257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:37.439978 14200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:37.440981 14200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:40:37.446342 13616 heartbeater.cc:503] Master 127.5.222.254:43161 was elected leader, sending a full tablet report...
W20260809 04:40:37.451095 14206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:37.451076 14209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:37.451064 14205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:37.451436 14200 server_base.cc:1034] running on GCE node
I20260809 04:40:37.451851 14200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:37.452590 14200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:37.453764 14200 hybrid_clock.cc:648] HybridClock initialized: now 1786250437453751 us; error 36 us; skew 500 ppm
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:37.456688 14200 init.cc:377] Logged in from keytab as oryx/127.5.222.194@KRBTEST.COM (short username oryx)
I20260809 04:40:37.457556 14200 webserver.cc:460] Webserver started at http://127.5.222.194:46227/ using document root <none> and password file <none>
I20260809 04:40:37.458179 14200 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:37.458272 14200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:37.462476 14200 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:40:37.464382 14217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:37.465611 14200 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:37.465726 14200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "0b660677db014b769e872ce5abb04084"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "c676741c6fd0047e74006798496b8863"
server_key_iv: "95ec0fece0aaa4c0ac1df9c56b54da57"
server_key_version: "encryptionkey@0"
I20260809 04:40:37.466235 14200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:37.475100 14200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:37.478084 14200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:37.478298 14200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:37.478907 14200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:37.479760 14200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:37.479809 14200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:37.479876 14200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:37.479905 14200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:37.491673 14200 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:43927
I20260809 04:40:37.491796 14351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:43927 every 8 connection(s)
I20260809 04:40:37.492666 14200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:37.498806 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 14200
I20260809 04:40:37.498970 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 13472
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.253@KRBTEST.COM
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:37.513084 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.496349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:33911 (local address 127.5.222.253:44613)
0809 04:40:37.496473 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.496478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.496621 (+ 143us) server_negotiation.cc:407] Connection header received
0809 04:40:37.497079 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.497082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.497142 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.497244 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.498414 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.499205 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.504324 (+ 5119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.504746 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.504870 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.508629 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.508635 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.508640 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.508671 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.510432 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.511078 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.511082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.511085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.511185 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.511665 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.511671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.511676 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.511937 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.512018 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.512774 (+ 756us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.512909 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.252@KRBTEST.COM
I20260809 04:40:37.513567 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.494633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:57709 (local address 127.5.222.254:43161)
0809 04:40:37.494759 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.494763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.495900 (+ 1137us) server_negotiation.cc:407] Connection header received
0809 04:40:37.497105 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.497109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.497193 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.497278 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.498434 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.499256 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.504865 (+ 5609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.505277 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.505385 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.508698 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.508705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.508711 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.508745 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.511139 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.511890 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.511896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.511901 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.511990 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.512343 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.512349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.512354 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.512690 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.512772 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.513175 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.513321 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:40:37.514796 14357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.495196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.194:57709)
0809 04:40:37.495727 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.495789 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.496868 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.497431 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.497433 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.497849 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.498281 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.498287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.503432 (+ 5145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.503438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.504725 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.504739 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.505590 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.505596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.505745 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.506429 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.506468 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.508502 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.511292 (+ 2790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.511298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.511303 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.511770 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.512104 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.512110 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.512115 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.512211 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.512849 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.512855 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.512959 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.514157 (+ 1198us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.514345 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":118,"spinlock_wait_cycles":52224,"thread_start_us":173,"threads_started":1}
I20260809 04:40:37.515069 14358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.496241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:44613 (local address 127.5.222.194:33911)
0809 04:40:37.496515 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.496526 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.496878 (+ 352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.497402 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.497417 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.497780 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.498265 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.498277 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.503229 (+ 4952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.503232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.504157 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.504165 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.505085 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.505091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.505338 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.505976 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.506000 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.508457 (+ 2457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.510563 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.510569 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.510583 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.510946 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.511312 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.511317 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.511320 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.511546 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.512103 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.512113 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.512431 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.513291 (+ 860us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.513640 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":2,"spinlock_wait_cycles":49664,"thread_start_us":99,"threads_started":1}
I20260809 04:40:37.515700 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:38385
--local_ip_for_outbound_sockets=127.5.222.195
--tserver_master_addrs=127.5.222.254:43161
--webserver_port=40439
--webserver_interface=127.5.222.195
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab
--principal=oryx/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.222.254:43161,127.5.222.253:44613
--tserver_master_addrs=127.5.222.254:43161,127.5.222.253:44613,127.5.222.252:35257 with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:40:37.516623 14354 heartbeater.cc:347] Connected to a master server at 127.5.222.253:44613
I20260809 04:40:37.516860 14354 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:37.517467 14354 heartbeater.cc:511] Master 127.5.222.253:44613 requested a full tablet report, sending...
I20260809 04:40:37.517589 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.494628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:53679 (local address 127.5.222.252:35257)
0809 04:40:37.494817 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.494822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.495900 (+ 1078us) server_negotiation.cc:407] Connection header received
0809 04:40:37.497105 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.497109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.497193 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.497250 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.498441 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.499249 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.510075 (+ 10826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.510491 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.510568 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.513653 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.513657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.513660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.513687 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.515404 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.516080 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.516086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.516091 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.516182 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.516488 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.516491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.516493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.516737 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.516812 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.517233 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.517385 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:40:37.518117 14353 heartbeater.cc:347] Connected to a master server at 127.5.222.254:43161
I20260809 04:40:37.518213 14353 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:37.518460 14353 heartbeater.cc:511] Master 127.5.222.254:43161 requested a full tablet report, sending...
I20260809 04:40:37.518201 14356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.495196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:35257 (local address 127.5.222.194:53679)
0809 04:40:37.495727 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.495796 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.496867 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.497411 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.497421 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.497849 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.498278 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.498287 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.508917 (+ 10630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.508923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.509917 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.509930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.510741 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.510745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.510863 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.511356 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.511370 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.513491 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.515688 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.515691 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.515692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.515960 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.516297 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.516299 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.516301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.516386 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.516882 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.516884 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.517007 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.517801 (+ 794us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.517974 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":114,"spinlock_wait_cycles":18944,"thread_start_us":122,"threads_started":1}
I20260809 04:40:37.518822 13782 ts_manager.cc:194] Registered new tserver with Master: 0b660677db014b769e872ce5abb04084 (127.5.222.194:43927)
I20260809 04:40:37.519156 13080 ts_manager.cc:194] Re-registered known tserver with Master: 0b660677db014b769e872ce5abb04084 (127.5.222.194:43927)
I20260809 04:40:37.519381 14352 heartbeater.cc:347] Connected to a master server at 127.5.222.252:35257
I20260809 04:40:37.519464 14352 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:37.519604 14352 heartbeater.cc:511] Master 127.5.222.252:35257 requested a full tablet report, sending...
I20260809 04:40:37.519959 13080 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:57709
I20260809 04:40:37.520323 13988 ts_manager.cc:194] Registered new tserver with Master: 0b660677db014b769e872ce5abb04084 (127.5.222.194:43927)
W20260809 04:40:37.629819 14359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:37.630059 14359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:37.630100 14359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:37.633894 14359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:37.633941 14359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:37.633994 14359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:37.639070 14359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab
--principal=oryx/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:38385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=40439
--tserver_master_addrs=127.5.222.254:43161,127.5.222.253:44613,127.5.222.252:35257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:37.639972 14359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:37.640893 14359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:37.650357 14365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:37.650427 14368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:37.650357 14366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:37.650616 14359 server_base.cc:1034] running on GCE node
I20260809 04:40:37.651044 14359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:37.651682 14359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:37.652855 14359 hybrid_clock.cc:648] HybridClock initialized: now 1786250437652830 us; error 37 us; skew 500 ppm
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:37.656791 14359 init.cc:377] Logged in from keytab as oryx/127.5.222.195@KRBTEST.COM (short username oryx)
I20260809 04:40:37.656886 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.652286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:35257 (local address 127.0.0.1:40548)
0809 04:40:37.652528 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.652546 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.652659 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.653082 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.653085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.653109 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:37.653268 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.653279 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.654455 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.654462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.655464 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.655475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.656443 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.656447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.656545 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.656570 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.656655 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.656739 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":82,"threads_started":1}
I20260809 04:40:37.656883 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.652378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40548 (local address 127.5.222.252:35257)
0809 04:40:37.652632 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.652637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.652652 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:40:37.652803 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.652806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.652878 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.652997 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:37.653414 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.654345 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.655625 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.656276 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.656420 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.656573 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.656669 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.656738 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260809 04:40:37.657858 14359 webserver.cc:460] Webserver started at http://127.5.222.195:40439/ using document root <none> and password file <none>
I20260809 04:40:37.657838 13999 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 04:40:37.658267 14036 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 086656952f8644e4a572db00a1becc1f. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.658495 14036 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:40:37.658576 14359 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:37.658684 14359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:37.662557 14359 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:40:37.664502 14377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:37.665695 14359 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:37.665861 14359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "1a56bf57806d4996bf4de69813ea10d6"
format_stamp: "Formatted at 2026-08-09 04:40:36 on dist-test-slave-vxj2"
server_key: "e8f45d3a0a9c3a4422e2203e2cd9f32f"
server_key_iv: "163106bc00fdc3bbdd868728b2755c70"
server_key_version: "encryptionkey@0"
I20260809 04:40:37.666476 14359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:37.699765 14359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:37.703526 14359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:37.703693 14359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:37.704260 14359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:37.705088 14359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:37.705137 14359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:37.705235 14359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:37.705271 14359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:37.715133 14359 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:38385
I20260809 04:40:37.715159 14509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:38385 every 8 connection(s)
I20260809 04:40:37.716145 14359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:37.723881 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 14359
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.252@KRBTEST.COM
Aug 09 04:40:37 dist-test-slave-vxj2 krb5kdc[13028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.253@KRBTEST.COM
I20260809 04:40:37.735488 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.718170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:39019 (local address 127.5.222.254:43161)
0809 04:40:37.718385 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.718391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.719498 (+ 1107us) server_negotiation.cc:407] Connection header received
0809 04:40:37.720797 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.720801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.720862 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.720922 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.722441 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.723026 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.725234 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.725615 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.725754 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.730285 (+ 4531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.730290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.730294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.730332 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.732249 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.733061 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.733065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.733067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.733199 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.733664 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.733667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.733669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.733884 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.733978 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.735067 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.735210 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 04:40:37.737004 14515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.718721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:35257 (local address 127.5.222.195:40589)
0809 04:40:37.719304 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.719375 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.720538 (+ 1163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.721081 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.721091 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.721654 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.722267 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.722277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.723295 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.723298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.725088 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.725099 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.726003 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.726008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.726239 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.726959 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.726982 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.729912 (+ 2930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.732380 (+ 2468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.732387 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.732404 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.732940 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.733312 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.733316 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.733318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.733544 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.734038 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.734045 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.735247 (+ 1202us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.735380 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.735780 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":500,"spinlock_wait_cycles":1853824,"thread_start_us":141,"threads_started":1}
I20260809 04:40:37.737764 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.718155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:40589 (local address 127.5.222.252:35257)
0809 04:40:37.718517 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.718524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.719494 (+ 970us) server_negotiation.cc:407] Connection header received
0809 04:40:37.720797 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.720801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.720863 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.720922 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.722384 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.723183 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.725275 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.725643 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.725755 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.730110 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.730114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.730117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.730182 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.732238 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.733061 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.733065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.733067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.733201 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.733664 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.733667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.733669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.733884 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.733978 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.737448 (+ 3470us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.737590 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238}
I20260809 04:40:37.738277 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.718721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.195:39019)
0809 04:40:37.719303 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.719375 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.720538 (+ 1163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.721081 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.721092 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.721643 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.722267 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.722277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.723185 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.723189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.725074 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.725087 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.725951 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.725956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.726620 (+ 664us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.727109 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.727127 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.730087 (+ 2960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.732380 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.732387 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.732404 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.732940 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.733312 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.733316 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.733318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.733544 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.734038 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.734045 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.734598 (+ 553us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.737906 (+ 3308us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.738054 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":942,"spinlock_wait_cycles":382976,"thread_start_us":158,"threads_started":1}
I20260809 04:40:37.738977 14511 heartbeater.cc:347] Connected to a master server at 127.5.222.254:43161
I20260809 04:40:37.739241 14511 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:37.739874 14511 heartbeater.cc:511] Master 127.5.222.254:43161 requested a full tablet report, sending...
I20260809 04:40:37.740484 14510 heartbeater.cc:347] Connected to a master server at 127.5.222.252:35257
I20260809 04:40:37.740576 14510 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:37.740768 14510 heartbeater.cc:511] Master 127.5.222.252:35257 requested a full tablet report, sending...
I20260809 04:40:37.741636 13080 ts_manager.cc:194] Re-registered known tserver with Master: 1a56bf57806d4996bf4de69813ea10d6 (127.5.222.195:38385)
I20260809 04:40:37.741727 13988 ts_manager.cc:194] Registered new tserver with Master: 1a56bf57806d4996bf4de69813ea10d6 (127.5.222.195:38385)
I20260809 04:40:37.742479 13080 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:39019
I20260809 04:40:37.742888 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.719927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:55759 (local address 127.5.222.253:44613)
0809 04:40:37.720053 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.720059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.720195 (+ 136us) server_negotiation.cc:407] Connection header received
0809 04:40:37.720768 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.720772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.720828 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.720898 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.722294 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.723110 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.725360 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.725695 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.725780 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.734700 (+ 8920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.734704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.734707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.734731 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.738009 (+ 3278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.741514 (+ 3505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.741518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.741520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.741598 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.741930 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.741933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.741935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.742173 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.742265 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.742613 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.742732 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:40:37.743311 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.726156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55344 (local address 127.5.222.254:43161)
0809 04:40:37.729961 (+ 3805us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.729965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.729981 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:40:37.730026 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.730030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.730083 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.730882 (+ 799us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.731325 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.732309 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.736011 (+ 3702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.736493 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.737004 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.738325 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.738331 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.738337 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.738366 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.740603 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.741260 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.741265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.741270 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.741369 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.741827 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.741830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.741832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.742060 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.742735 (+ 675us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.742826 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.743149 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3722,"thread_start_us":81,"threads_started":1}
I20260809 04:40:37.744307 14525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.726484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53910 (local address 127.5.222.253:44613)
0809 04:40:37.726749 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.726754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.728660 (+ 1906us) server_negotiation.cc:407] Connection header received
0809 04:40:37.728802 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.728806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.728866 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.730509 (+ 1643us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.731664 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.732553 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.733886 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.734274 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.734375 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.737000 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.737004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.737006 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.737035 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.740466 (+ 3431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.741053 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.741059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.741063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.741198 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.742759 (+ 1561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.742766 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.742794 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.743062 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.743192 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.743953 (+ 761us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.744103 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
I20260809 04:40:37.746124 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.719831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:44613 (local address 127.5.222.195:55759)
0809 04:40:37.720097 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.720109 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.720550 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.721080 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.721087 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.721610 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:37.722138 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.722153 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.723262 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.723265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.725246 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.725253 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.726509 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.726513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.727156 (+ 643us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.727643 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:37.727663 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:37.734531 (+ 6868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:37.738125 (+ 3594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.738128 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.738130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.741400 (+ 3270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.741714 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:37.741717 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:37.741720 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:37.741815 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:37.742329 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:37.742332 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:37.742425 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.745064 (+ 2639us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.745898 (+ 834us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":274,"thread_start_us":100,"threads_started":1}
I20260809 04:40:37.746930 14513 heartbeater.cc:347] Connected to a master server at 127.5.222.253:44613
I20260809 04:40:37.747026 14513 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:37.747248 14513 heartbeater.cc:511] Master 127.5.222.253:44613 requested a full tablet report, sending...
I20260809 04:40:37.747628 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.727017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.5.222.252:35257)
0809 04:40:37.727818 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.727831 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.729228 (+ 1397us) server_negotiation.cc:407] Connection header received
0809 04:40:37.729375 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.729378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.729428 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.729716 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:37.732669 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.733607 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.734947 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.735353 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.735458 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.740759 (+ 5301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:37.740763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:37.740766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:37.740795 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:37.744492 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.745590 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.745594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.745596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.745689 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:37.746001 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:37.746005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:37.746007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:37.746457 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:37.746599 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.747205 (+ 606us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.747429 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":119,"threads_started":1}
I20260809 04:40:37.747992 13782 ts_manager.cc:194] Registered new tserver with Master: 1a56bf57806d4996bf4de69813ea10d6 (127.5.222.195:38385)
I20260809 04:40:37.747862 13080 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:37.751426 13080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:37.754091 14371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: NON_VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: true } } }
I20260809 04:40:37.754236 14371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:37.757560 14531 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f: attempting to promote NON_VOTER c39ed4367f384162a78b1aafad4cde45 to VOTER
I20260809 04:40:37.757958 14531 consensus_queue.cc:237] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } }
I20260809 04:40:37.759176 13791 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 FOLLOWER]: Refusing update from remote peer 086656952f8644e4a572db00a1becc1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 04:40:37.759328 13998 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 1 LEARNER]: Refusing update from remote peer 086656952f8644e4a572db00a1becc1f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 04:40:37.759591 14371 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 04:40:37.759900 14371 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 04:40:37.762023 14531 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, c39ed4367f384162a78b1aafad4cde45 (127.5.222.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } } }
I20260809 04:40:37.762480 13791 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, c39ed4367f384162a78b1aafad4cde45 (127.5.222.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } } }
I20260809 04:40:37.763593 14531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } } }
I20260809 04:40:37.763732 14531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:37.763890 14531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } } }
I20260809 04:40:37.763998 14531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:37.764603 14531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } } }
I20260809 04:40:37.764731 14531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:37.764750 14542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } } }
I20260809 04:40:37.764885 14542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:40:37.765396 13998 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, c39ed4367f384162a78b1aafad4cde45 (127.5.222.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } } }
I20260809 04:40:37.767537 14036 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "086656952f8644e4a572db00a1becc1f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } } }
I20260809 04:40:37.767650 14036 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:40:37.773898 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.768216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:38385 (local address 127.0.0.1:34144)
0809 04:40:37.768417 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.768433 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.768524 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.769634 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.769636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.769656 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:37.769769 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.769776 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.771331 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.771343 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.772440 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.772450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.773499 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.773502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.773589 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.773611 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.773691 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.773770 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260809 04:40:37.773950 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.768379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34144 (local address 127.5.222.195:38385)
0809 04:40:37.768817 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.768830 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.768855 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:40:37.768934 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.768941 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.769210 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.769677 (+ 467us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:37.769931 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.771220 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.772544 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.773373 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.773489 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.773639 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.773720 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.773804 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":251,"threads_started":1}
I20260809 04:40:37.776105 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.768870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:38087 (local address 127.0.0.1:38858)
0809 04:40:37.769934 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.769948 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.770180 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.771603 (+ 1423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.771607 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.771626 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:37.771751 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.771758 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.773064 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.773067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.774374 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.774386 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.775634 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.775638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.775759 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.775788 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.775876 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.775948 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1000,"thread_start_us":95,"threads_started":1}
I20260809 04:40:37.776741 14549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.768875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43927 (local address 127.0.0.1:44350)
0809 04:40:37.770389 (+ 1514us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.770403 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.770712 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.771884 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.771886 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.771903 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:37.771974 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.771978 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.773286 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.773289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.774880 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.774890 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.776098 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.776105 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:37.776402 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.776434 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:40:37.776529 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 04:40:37.776599 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1454,"mutex_wait_us":122,"thread_start_us":93,"threads_started":1}
I20260809 04:40:37.776841 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.769672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44350 (local address 127.5.222.194:43927)
0809 04:40:37.770964 (+ 1292us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.770978 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.771004 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:40:37.771093 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.771099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.771297 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.771626 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:37.772086 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.773133 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.775010 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.775808 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.775902 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.776027 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.776561 (+ 534us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.776648 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1196,"thread_start_us":142,"threads_started":1}
I20260809 04:40:37.776935 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.769212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38858 (local address 127.5.222.193:38087)
0809 04:40:37.770971 (+ 1759us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.770987 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.771006 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:40:37.771150 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.771154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.771296 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.771626 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:37.771855 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.772933 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.774527 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:37.775343 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:37.775440 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:37.775568 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:37.775925 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 04:40:37.776022 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1676,"thread_start_us":154,"threads_started":1}
I20260809 04:40:37.777089 14433 tablet_service.cc:1467] Processing CreateTablet for tablet b4db1483448b4305b8fcd5143506226c (DEFAULT_TABLE table=test-table [id=b5df4943783d4bc4a6c736abca2d60a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:37.778239 14433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4db1483448b4305b8fcd5143506226c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:37.779377 14270 tablet_service.cc:1467] Processing CreateTablet for tablet b4db1483448b4305b8fcd5143506226c (DEFAULT_TABLE table=test-table [id=b5df4943783d4bc4a6c736abca2d60a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:37.779579 14116 tablet_service.cc:1467] Processing CreateTablet for tablet b4db1483448b4305b8fcd5143506226c (DEFAULT_TABLE table=test-table [id=b5df4943783d4bc4a6c736abca2d60a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:37.780257 14270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4db1483448b4305b8fcd5143506226c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:37.780550 14116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4db1483448b4305b8fcd5143506226c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:37.787762 14553 tablet_bootstrap.cc:492] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6: Bootstrap starting.
I20260809 04:40:37.788170 14555 tablet_bootstrap.cc:492] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084: Bootstrap starting.
I20260809 04:40:37.789007 14557 tablet_bootstrap.cc:492] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0: Bootstrap starting.
I20260809 04:40:37.790071 14553 tablet_bootstrap.cc:654] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:37.790659 14555 tablet_bootstrap.cc:654] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:37.790869 14553 log.cc:824] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:37.791499 14555 log.cc:824] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:37.791908 14557 tablet_bootstrap.cc:654] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:37.792691 14557 log.cc:824] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:37.792948 14553 tablet_bootstrap.cc:492] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6: No bootstrap required, opened a new log
I20260809 04:40:37.793195 14553 ts_tablet_manager.cc:1397] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:40:37.794195 14555 tablet_bootstrap.cc:492] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084: No bootstrap required, opened a new log
I20260809 04:40:37.794399 14555 ts_tablet_manager.cc:1397] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 04:40:37.794696 14557 tablet_bootstrap.cc:492] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0: No bootstrap required, opened a new log
I20260809 04:40:37.794883 14557 ts_tablet_manager.cc:1397] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:40:37.797892 14555 raft_consensus.cc:348] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } }
I20260809 04:40:37.798128 14555 raft_consensus.cc:374] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:37.797995 14553 raft_consensus.cc:348] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } }
I20260809 04:40:37.798189 14555 raft_consensus.cc:729] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b660677db014b769e872ce5abb04084, State: Initialized, Role: FOLLOWER
I20260809 04:40:37.798239 14553 raft_consensus.cc:374] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:37.798290 14553 raft_consensus.cc:729] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a56bf57806d4996bf4de69813ea10d6, State: Initialized, Role: FOLLOWER
I20260809 04:40:37.798707 14555 consensus_queue.cc:260] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } }
I20260809 04:40:37.798733 14553 consensus_queue.cc:260] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } }
I20260809 04:40:37.799418 14353 heartbeater.cc:503] Master 127.5.222.254:43161 was elected leader, sending a full tablet report...
I20260809 04:40:37.799463 14557 raft_consensus.cc:348] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } }
I20260809 04:40:37.799685 14557 raft_consensus.cc:374] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:37.799736 14557 raft_consensus.cc:729] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8750546d7c214df3902dfe9a782667c0, State: Initialized, Role: FOLLOWER
I20260809 04:40:37.800156 14557 consensus_queue.cc:260] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } }
I20260809 04:40:37.800808 14191 heartbeater.cc:503] Master 127.5.222.254:43161 was elected leader, sending a full tablet report...
I20260809 04:40:37.801280 14553 ts_tablet_manager.cc:1428] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 04:40:37.802085 14557 ts_tablet_manager.cc:1428] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 04:40:37.802642 14511 heartbeater.cc:503] Master 127.5.222.254:43161 was elected leader, sending a full tablet report...
I20260809 04:40:37.807264 14555 ts_tablet_manager.cc:1428] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
W20260809 04:40:37.967437 14514 tablet.cc:2369] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:40:37.995137 14355 tablet.cc:2369] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:40:37.996974 14565 raft_consensus.cc:482] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:37.997227 14565 raft_consensus.cc:504] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } }
I20260809 04:40:37.998467 14565 leader_election.cc:288] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a56bf57806d4996bf4de69813ea10d6 (127.5.222.195:38385), 0b660677db014b769e872ce5abb04084 (127.5.222.194:43927)
I20260809 04:40:38.002943 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.998897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:41757 (local address 127.5.222.194:43927)
0809 04:40:37.999028 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.999033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.999458 (+ 425us) server_negotiation.cc:407] Connection header received
0809 04:40:37.999606 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.999609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.999675 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.999748 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:38.000042 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.000612 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.001907 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.002343 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.002480 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:38.002532 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:38.002768 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 04:40:38.002844 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:40:38.002954 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.998897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:57263 (local address 127.5.222.195:38385)
0809 04:40:37.999028 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:37.999033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:37.999382 (+ 349us) server_negotiation.cc:407] Connection header received
0809 04:40:37.999454 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:37.999458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:37.999512 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:37.999598 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:37.999929 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.000495 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.001852 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.002276 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.002406 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:38.002483 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:38.002776 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 04:40:38.002855 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 04:40:38.002980 14566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.998688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43927 (local address 127.5.222.193:41757)
0809 04:40:37.999172 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.999185 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.999471 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.999772 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.999774 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.999789 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:37.999885 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.999891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.000834 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.000838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:38.001764 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:38.001777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.002473 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.002477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:38.002624 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:38.002643 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 04:40:38.002729 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:40:38.002803 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":60,"thread_start_us":230,"threads_started":1}
I20260809 04:40:38.003365 14568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:37.998688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:38385 (local address 127.5.222.193:57263)
0809 04:40:37.999251 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:37.999263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:37.999359 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:37.999616 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:37.999619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:37.999636 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:37.999793 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:37.999800 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.000636 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.000642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:38.001717 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:38.001727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.002456 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.002462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:38.002630 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:38.002653 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:40:38.003126 (+ 473us) client_negotiation.cc:240] Negotiation successful
0809 04:40:38.003222 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":175,"thread_start_us":68,"threads_started":1}
I20260809 04:40:38.003705 14298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4db1483448b4305b8fcd5143506226c" candidate_uuid: "8750546d7c214df3902dfe9a782667c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b660677db014b769e872ce5abb04084" is_pre_election: true
I20260809 04:40:38.004004 14298 raft_consensus.cc:2454] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8750546d7c214df3902dfe9a782667c0 in term 0.
I20260809 04:40:38.003980 14456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4db1483448b4305b8fcd5143506226c" candidate_uuid: "8750546d7c214df3902dfe9a782667c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a56bf57806d4996bf4de69813ea10d6" is_pre_election: true
I20260809 04:40:38.004235 14456 raft_consensus.cc:2454] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8750546d7c214df3902dfe9a782667c0 in term 0.
I20260809 04:40:38.004676 14066 leader_election.cc:302] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b660677db014b769e872ce5abb04084, 8750546d7c214df3902dfe9a782667c0; no voters:
I20260809 04:40:38.004951 14565 raft_consensus.cc:2781] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:38.005046 14565 raft_consensus.cc:482] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:38.005105 14565 raft_consensus.cc:3037] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:38.006181 14565 raft_consensus.cc:504] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } }
I20260809 04:40:38.006559 14565 leader_election.cc:288] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [CANDIDATE]: Term 1 election: Requested vote from peers 1a56bf57806d4996bf4de69813ea10d6 (127.5.222.195:38385), 0b660677db014b769e872ce5abb04084 (127.5.222.194:43927)
I20260809 04:40:38.006927 14298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4db1483448b4305b8fcd5143506226c" candidate_uuid: "8750546d7c214df3902dfe9a782667c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b660677db014b769e872ce5abb04084"
I20260809 04:40:38.006949 14456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4db1483448b4305b8fcd5143506226c" candidate_uuid: "8750546d7c214df3902dfe9a782667c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a56bf57806d4996bf4de69813ea10d6"
I20260809 04:40:38.007036 14298 raft_consensus.cc:3037] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:38.007083 14456 raft_consensus.cc:3037] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:38.008102 14456 raft_consensus.cc:2454] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8750546d7c214df3902dfe9a782667c0 in term 1.
I20260809 04:40:38.008118 14298 raft_consensus.cc:2454] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8750546d7c214df3902dfe9a782667c0 in term 1.
I20260809 04:40:38.008415 14062 leader_election.cc:302] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a56bf57806d4996bf4de69813ea10d6, 8750546d7c214df3902dfe9a782667c0; no voters:
I20260809 04:40:38.008610 14565 raft_consensus.cc:2781] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:38.008877 14565 raft_consensus.cc:686] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 1 LEADER]: Becoming Leader. State: Replica: 8750546d7c214df3902dfe9a782667c0, State: Running, Role: LEADER
I20260809 04:40:38.009261 14565 consensus_queue.cc:237] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } }
I20260809 04:40:38.012800 13080 catalog_manager.cc:5495] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8750546d7c214df3902dfe9a782667c0 (127.5.222.193). New cstate: current_term: 1 leader_uuid: "8750546d7c214df3902dfe9a782667c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } health_report { overall_health: UNKNOWN } } }
W20260809 04:40:38.038651 14193 tablet.cc:2369] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:40:38.064075 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:38.060375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38862 (local address 127.5.222.193:38087)
0809 04:40:38.060519 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:38.060523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:38.060539 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:40:38.060693 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:38.060696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:38.060750 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:38.060842 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:38.061145 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.061707 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.062684 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.062887 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.062972 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:38.063325 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:38.063721 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:38.063810 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:38.063916 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 04:40:38.063967 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:40:38.069800 14298 raft_consensus.cc:1261] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084 [term 1 FOLLOWER]: Refusing update from remote peer 8750546d7c214df3902dfe9a782667c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:38.069800 14456 raft_consensus.cc:1261] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6 [term 1 FOLLOWER]: Refusing update from remote peer 8750546d7c214df3902dfe9a782667c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:38.070572 14569 consensus_queue.cc:1035] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:38.070848 14565 consensus_queue.cc:1035] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:38.077883 14573 mvcc.cc:205] Tried to move back new op lower bound from 7316481794326724608 to 7316481794088538112. Current Snapshot: MvccSnapshot[applied={T|T < 7316481794326724608}]
I20260809 04:40:38.080953 14575 mvcc.cc:205] Tried to move back new op lower bound from 7316481794326724608 to 7316481794088538112. Current Snapshot: MvccSnapshot[applied={T|T < 7316481794326724608}]
I20260809 04:40:38.090759 13081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:38.090903 13081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:38.095253 13081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 086656952f8644e4a572db00a1becc1f: Sending DeleteTablet for 3 replicas of tablet b4db1483448b4305b8fcd5143506226c
I20260809 04:40:38.096007 14116 tablet_service.cc:1514] Processing DeleteTablet for tablet b4db1483448b4305b8fcd5143506226c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:38 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:38858
I20260809 04:40:38.096174 14433 tablet_service.cc:1514] Processing DeleteTablet for tablet b4db1483448b4305b8fcd5143506226c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:38 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:34144
I20260809 04:40:38.096587 14584 tablet_replica.cc:330] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0: stopping tablet replica
I20260809 04:40:38.096692 14586 tablet_replica.cc:330] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6: stopping tablet replica
I20260809 04:40:38.096684 14270 tablet_service.cc:1514] Processing DeleteTablet for tablet b4db1483448b4305b8fcd5143506226c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:38 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:44350
I20260809 04:40:38.097013 14586 raft_consensus.cc:2229] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:38.097020 14584 raft_consensus.cc:2229] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:38.097368 14586 raft_consensus.cc:2258] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:38.097425 14584 raft_consensus.cc:2258] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:38.098194 14587 tablet_replica.cc:330] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084: stopping tablet replica
I20260809 04:40:38.098515 14587 raft_consensus.cc:2229] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:38.098837 14587 raft_consensus.cc:2258] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:38.099004 14586 ts_tablet_manager.cc:1905] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:38.100505 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 13043
I20260809 04:40:38.100512 14587 ts_tablet_manager.cc:1905] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:38.103576 14584 ts_tablet_manager.cc:1905] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:38.105865 14587 ts_tablet_manager.cc:1918] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:38.106019 14587 log.cc:1196] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/b4db1483448b4305b8fcd5143506226c
I20260809 04:40:38.106415 14587 ts_tablet_manager.cc:1939] T b4db1483448b4305b8fcd5143506226c P 0b660677db014b769e872ce5abb04084: Deleting consensus metadata
I20260809 04:40:38.107993 14586 ts_tablet_manager.cc:1918] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:38.108110 14586 log.cc:1196] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/b4db1483448b4305b8fcd5143506226c
I20260809 04:40:38.108477 14586 ts_tablet_manager.cc:1939] T b4db1483448b4305b8fcd5143506226c P 1a56bf57806d4996bf4de69813ea10d6: Deleting consensus metadata
W20260809 04:40:38.111138 14220 connection.cc:535] server connection from 127.0.0.1:44350 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 04:40:38.111812 14379 connection.cc:535] server connection from 127.0.0.1:34144 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 04:40:38.112129 14584 ts_tablet_manager.cc:1918] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:38.112264 14584 log.cc:1196] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-0/wal/wals/b4db1483448b4305b8fcd5143506226c
I20260809 04:40:38.112658 14584 ts_tablet_manager.cc:1939] T b4db1483448b4305b8fcd5143506226c P 8750546d7c214df3902dfe9a782667c0: Deleting consensus metadata
W20260809 04:40:38.113719 14062 connection.cc:441] server connection from 127.0.0.1:38858 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38858 (request call id 1) could send its response
I20260809 04:40:38.120898 14525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:38.099176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53924 (local address 127.5.222.253:44613)
0809 04:40:38.099286 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:38.099290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:38.107945 (+ 8655us) server_negotiation.cc:407] Connection header received
0809 04:40:38.108259 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:38.108263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:38.108311 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:38.108361 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:38.109455 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.110295 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.111582 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.111942 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.112012 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:38.114322 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:38.114326 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:38.114329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:38.114357 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:38.116394 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:38.116950 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:38.116954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:38.116957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:38.117063 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:38.117440 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:38.117445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:38.117448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:38.117701 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:38.117864 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:38.120612 (+ 2748us) server_negotiation.cc:299] Negotiation successful
0809 04:40:38.120738 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:40:38.127897 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:38.115644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53936 (local address 127.5.222.253:44613)
0809 04:40:38.115795 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:38.115802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:38.116569 (+ 767us) server_negotiation.cc:407] Connection header received
0809 04:40:38.116730 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:38.116733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:38.116775 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:38.116822 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:38.117893 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.118693 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.119888 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.120228 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.120296 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:38.123219 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:38.123224 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:38.123230 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:38.123260 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:38.125675 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:38.126214 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:38.126221 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:38.126227 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:38.126335 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:38.126663 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:38.126666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:38.126668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:38.126943 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:38.127102 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:38.127576 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 04:40:38.127707 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:40:38.128780 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:38.116379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40576 (local address 127.5.222.252:35257)
0809 04:40:38.116515 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:38.116520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:38.116940 (+ 420us) server_negotiation.cc:407] Connection header received
0809 04:40:38.117189 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:38.117194 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:38.117253 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:38.117319 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:38.118382 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.119319 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.121108 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.121580 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.121661 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:38.124574 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:38.124578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:38.124580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:38.124605 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:38.126770 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:38.127236 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:38.127240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:38.127242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:38.127325 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:38.127625 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:38.127628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:38.127630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:38.127826 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:38.127937 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:38.128512 (+ 575us) server_negotiation.cc:299] Negotiation successful
0809 04:40:38.128637 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:40:38.130517 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:38.098966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40562 (local address 127.5.222.252:35257)
0809 04:40:38.099089 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:38.099094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:38.118661 (+ 19567us) server_negotiation.cc:407] Connection header received
0809 04:40:38.118700 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:38.118703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:38.118749 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:38.118822 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:38.119728 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.120590 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.122153 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:38.122636 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:38.122779 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:38.125070 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:38.125075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:38.125078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:38.125110 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:38.128263 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:38.129032 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:38.129035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:38.129037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:38.129131 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:38.129653 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:38.129656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:38.129658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:38.129848 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:38.129987 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:38.130307 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 04:40:38.130411 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 04:40:39.082796 14596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.082259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.195:43283)
0809 04:40:39.082568 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.082664 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":91,"threads_started":1}
W20260809 04:40:39.083168 14511 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:43161 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
I20260809 04:40:39.085484 14597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.084968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.193:41997)
0809 04:40:39.085300 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.085387 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":70,"threads_started":1}
W20260809 04:40:39.085986 14191 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:43161 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
I20260809 04:40:39.086558 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.085829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.194:56637)
0809 04:40:39.086333 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.086429 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":72,"threads_started":1}
W20260809 04:40:39.086813 14353 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:43161 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
I20260809 04:40:39.595381 14600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 086656952f8644e4a572db00a1becc1f)
I20260809 04:40:39.595669 14600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } }
I20260809 04:40:39.595876 14601 raft_consensus.cc:482] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 086656952f8644e4a572db00a1becc1f)
I20260809 04:40:39.596089 14601 raft_consensus.cc:504] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } }
I20260809 04:40:39.596762 14600 leader_election.cc:288] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 086656952f8644e4a572db00a1becc1f (127.5.222.254:43161), c39ed4367f384162a78b1aafad4cde45 (127.5.222.252:35257)
I20260809 04:40:39.597067 14601 leader_election.cc:288] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 086656952f8644e4a572db00a1becc1f (127.5.222.254:43161), 7d405a7019604bb88d8db6f3656676c4 (127.5.222.253:44613)
I20260809 04:40:39.597540 14602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.596965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55776)
0809 04:40:39.597341 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.597429 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":87,"threads_started":1}
W20260809 04:40:39.597848 13762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
I20260809 04:40:39.598001 14605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.597311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55784)
0809 04:40:39.597762 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.597845 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":69,"threads_started":1}
W20260809 04:40:39.598197 13970 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
I20260809 04:40:39.598943 14602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.598667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55798)
0809 04:40:39.598807 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.598865 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 04:40:39.599181 14605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.598919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55804)
0809 04:40:39.599064 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.599115 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260809 04:40:39.599148 13762 leader_election.cc:334] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 086656952f8644e4a572db00a1becc1f (127.5.222.254:43161): Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
W20260809 04:40:39.599299 13970 leader_election.cc:334] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 086656952f8644e4a572db00a1becc1f (127.5.222.254:43161): Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
I20260809 04:40:39.601946 14603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.596992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:35257 (local address 127.0.0.1:40580)
0809 04:40:39.597450 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.597467 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:39.597587 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:39.598283 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:39.598289 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:39.598316 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:39.598446 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.598455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.599628 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.599631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.600780 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.600790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.601636 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.601639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.601717 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.601737 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:40:39.601793 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 04:40:39.601846 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":5,"thread_start_us":67,"threads_started":1}
I20260809 04:40:39.602047 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.597671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40580 (local address 127.5.222.252:35257)
0809 04:40:39.597894 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:39.597900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:39.597917 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:40:39.597971 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:39.597974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:39.598025 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:39.598121 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:39.598598 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.599491 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.600919 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.601510 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.601616 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.601676 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:39.601835 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 04:40:39.601908 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":65,"threads_started":1}
I20260809 04:40:39.602469 13998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7d405a7019604bb88d8db6f3656676c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c39ed4367f384162a78b1aafad4cde45" is_pre_election: true
I20260809 04:40:39.602485 14604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.597452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:44613 (local address 127.0.0.1:53944)
0809 04:40:39.597781 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.597794 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:39.597886 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:39.598934 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:39.598938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:39.598960 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:39.599053 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.599060 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.600057 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.600061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.601208 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.601218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.602145 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.602150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.602238 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.602260 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:40:39.602310 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 04:40:39.602385 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":88,"threads_started":1}
I20260809 04:40:39.602715 13998 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d405a7019604bb88d8db6f3656676c4 in term 1.
I20260809 04:40:39.602906 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.598204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53944 (local address 127.5.222.253:44613)
0809 04:40:39.598503 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:39.598517 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:39.598533 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:40:39.598576 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:39.598581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:39.598624 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:39.599169 (+ 545us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:39.599282 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.599940 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.601344 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.601998 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.602589 (+ 591us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.602649 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:39.602695 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 04:40:39.602786 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":105,"threads_started":1}
I20260809 04:40:39.603104 13764 leader_election.cc:302] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d405a7019604bb88d8db6f3656676c4, c39ed4367f384162a78b1aafad4cde45; no voters: 086656952f8644e4a572db00a1becc1f
I20260809 04:40:39.603297 13791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c39ed4367f384162a78b1aafad4cde45" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7d405a7019604bb88d8db6f3656676c4" is_pre_election: true
I20260809 04:40:39.603351 14600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 04:40:39.603425 14600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 086656952f8644e4a572db00a1becc1f)
I20260809 04:40:39.603482 14600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:40:39.604534 14600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } }
I20260809 04:40:39.604779 13791 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c39ed4367f384162a78b1aafad4cde45 in current term 2: Already voted for candidate 7d405a7019604bb88d8db6f3656676c4 in this term.
I20260809 04:40:39.604943 14600 leader_election.cc:288] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [CANDIDATE]: Term 2 election: Requested vote from peers 086656952f8644e4a572db00a1becc1f (127.5.222.254:43161), c39ed4367f384162a78b1aafad4cde45 (127.5.222.252:35257)
I20260809 04:40:39.605350 14603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.605085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55816)
0809 04:40:39.605246 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.605296 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 04:40:39.605542 13973 leader_election.cc:302] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c39ed4367f384162a78b1aafad4cde45; no voters: 086656952f8644e4a572db00a1becc1f, 7d405a7019604bb88d8db6f3656676c4
I20260809 04:40:39.605651 13998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7d405a7019604bb88d8db6f3656676c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c39ed4367f384162a78b1aafad4cde45"
I20260809 04:40:39.605760 14601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:40:39.606024 14603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.605796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55826)
0809 04:40:39.605927 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.605979 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260809 04:40:39.606120 13762 leader_election.cc:334] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 086656952f8644e4a572db00a1becc1f (127.5.222.254:43161): Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
I20260809 04:40:39.606714 14601 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:40:39.607664 13998 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d405a7019604bb88d8db6f3656676c4 in term 2.
I20260809 04:40:39.608322 13764 leader_election.cc:302] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d405a7019604bb88d8db6f3656676c4, c39ed4367f384162a78b1aafad4cde45; no voters: 086656952f8644e4a572db00a1becc1f
I20260809 04:40:39.608494 14600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:40:39.608676 14600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [term 2 LEADER]: Becoming Leader. State: Replica: 7d405a7019604bb88d8db6f3656676c4, State: Running, Role: LEADER
I20260809 04:40:39.608916 14600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } }
I20260809 04:40:39.609999 14611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d405a7019604bb88d8db6f3656676c4. Latest consensus state: current_term: 2 leader_uuid: "7d405a7019604bb88d8db6f3656676c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } } }
I20260809 04:40:39.610095 14611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:39.610389 14615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:39.612720 14615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:39.613072 14615 catalog_manager.cc:1259] Loaded cluster ID: c2bfbc2767904c37bd7cfea1ce85703e
I20260809 04:40:39.613132 14615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:39.613785 14615 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:39.614177 14615 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: Loaded TSK: 0
I20260809 04:40:39.614825 14615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:40:39.649509 13781 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:39.652493 13781 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:39.655616 14603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.655383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55840)
0809 04:40:39.655498 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.655552 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 04:40:39.655733 13998 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [term 2 FOLLOWER]: Refusing update from remote peer 7d405a7019604bb88d8db6f3656676c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 04:40:39.656345 14603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.656124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.0.0.1:55846)
0809 04:40:39.656233 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.656278 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 04:40:39.656258 14600 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260809 04:40:39.656581 13762 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 -> Peer 086656952f8644e4a572db00a1becc1f (127.5.222.254:43161): Couldn't send request to peer 086656952f8644e4a572db00a1becc1f. Status: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 04:40:39.658070 14601 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d405a7019604bb88d8db6f3656676c4. Latest consensus state: current_term: 2 leader_uuid: "7d405a7019604bb88d8db6f3656676c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } } }
I20260809 04:40:39.658205 14601 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:40:39.659312 14600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "7d405a7019604bb88d8db6f3656676c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } } }
I20260809 04:40:39.659444 14600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:39.659816 14601 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "7d405a7019604bb88d8db6f3656676c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "086656952f8644e4a572db00a1becc1f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43161 } } peers { permanent_uuid: "7d405a7019604bb88d8db6f3656676c4" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 44613 } attrs { promote: false } } peers { permanent_uuid: "c39ed4367f384162a78b1aafad4cde45" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 35257 } attrs { promote: false } } }
I20260809 04:40:39.659942 14601 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39ed4367f384162a78b1aafad4cde45 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:40:39.669816 14603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.665388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43927 (local address 127.0.0.1:44362)
0809 04:40:39.665537 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.665554 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:39.665651 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:39.666062 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:39.666065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:39.666089 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:39.666164 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.666168 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.667213 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.667217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.668460 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.668470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.669415 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.669419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.669509 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.669536 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:40:39.669616 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 04:40:39.669688 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 04:40:39.669946 14602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.665997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:38087 (local address 127.0.0.1:38868)
0809 04:40:39.666247 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.666261 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:39.666353 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:39.666751 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:39.666754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:39.666772 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:39.666857 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.666862 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.667724 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.667730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.668490 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.668498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.669549 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.669553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.669660 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.669690 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 04:40:39.669771 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 04:40:39.669842 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186}
I20260809 04:40:39.669961 14623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.665459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44362 (local address 127.5.222.194:43927)
0809 04:40:39.665731 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:39.665736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:39.665748 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:40:39.665805 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:39.665807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:39.665844 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:39.665936 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:39.666279 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.667084 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.668601 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.669279 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.669419 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.669501 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:39.669705 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 04:40:39.669822 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":78,"threads_started":1}
I20260809 04:40:39.670465 14270 tablet_service.cc:1467] Processing CreateTablet for tablet be069dd1223e47b2990bcf9e93f46cd2 (DEFAULT_TABLE table=test-table [id=1222534f47af43fe9f2fdfb1d371481a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:39.670804 14270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be069dd1223e47b2990bcf9e93f46cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:39.670802 14624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.666238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38868 (local address 127.5.222.193:38087)
0809 04:40:39.666515 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:39.666519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:39.666535 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:40:39.666582 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:39.666585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:39.666627 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:39.666951 (+ 324us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:39.667047 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.667606 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.668653 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.669429 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.670478 (+ 1049us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.670531 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:39.670560 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 04:40:39.670657 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":91,"threads_started":1}
I20260809 04:40:39.671298 14116 tablet_service.cc:1467] Processing CreateTablet for tablet be069dd1223e47b2990bcf9e93f46cd2 (DEFAULT_TABLE table=test-table [id=1222534f47af43fe9f2fdfb1d371481a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:39.671566 14116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be069dd1223e47b2990bcf9e93f46cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:39.673638 14627 tablet_bootstrap.cc:492] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084: Bootstrap starting.
I20260809 04:40:39.674119 14630 tablet_bootstrap.cc:492] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0: Bootstrap starting.
I20260809 04:40:39.674523 14625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.666689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:38385 (local address 127.0.0.1:34156)
0809 04:40:39.667055 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.667067 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:39.667159 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:39.667520 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:39.667523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:39.667538 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:39.667616 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.667622 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.668626 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.668629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.670164 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.670173 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.674095 (+ 3922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.674100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.674199 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.674228 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 04:40:39.674324 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 04:40:39.674398 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":93,"threads_started":1}
I20260809 04:40:39.674736 14627 tablet_bootstrap.cc:654] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:39.674978 14626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.666800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34156 (local address 127.5.222.195:38385)
0809 04:40:39.667034 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:39.667038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:39.667121 (+ 83us) server_negotiation.cc:407] Connection header received
0809 04:40:39.667277 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:39.667281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:39.667330 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:39.667402 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:39.667739 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.668509 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.673257 (+ 4748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.673876 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.673978 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.674035 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:39.674686 (+ 651us) server_negotiation.cc:299] Negotiation successful
0809 04:40:39.674833 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":89,"threads_started":1}
I20260809 04:40:39.675215 14630 tablet_bootstrap.cc:654] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:39.675504 14433 tablet_service.cc:1467] Processing CreateTablet for tablet be069dd1223e47b2990bcf9e93f46cd2 (DEFAULT_TABLE table=test-table [id=1222534f47af43fe9f2fdfb1d371481a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:39.675834 14433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be069dd1223e47b2990bcf9e93f46cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:39.676316 14627 tablet_bootstrap.cc:492] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084: No bootstrap required, opened a new log
I20260809 04:40:39.676373 14630 tablet_bootstrap.cc:492] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0: No bootstrap required, opened a new log
I20260809 04:40:39.676414 14627 ts_tablet_manager.cc:1397] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:39.676452 14630 ts_tablet_manager.cc:1397] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:39.676853 14630 raft_consensus.cc:348] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } }
I20260809 04:40:39.676853 14627 raft_consensus.cc:348] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } }
I20260809 04:40:39.676973 14627 raft_consensus.cc:374] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:39.676973 14630 raft_consensus.cc:374] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:39.677009 14630 raft_consensus.cc:729] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8750546d7c214df3902dfe9a782667c0, State: Initialized, Role: FOLLOWER
I20260809 04:40:39.677009 14627 raft_consensus.cc:729] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b660677db014b769e872ce5abb04084, State: Initialized, Role: FOLLOWER
I20260809 04:40:39.677145 14627 consensus_queue.cc:260] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } }
I20260809 04:40:39.677212 14630 consensus_queue.cc:260] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } }
I20260809 04:40:39.677518 14627 ts_tablet_manager.cc:1428] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 04:40:39.677520 14630 ts_tablet_manager.cc:1428] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:39.679322 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.678335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.194:42905)
0809 04:40:39.679173 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.679235 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":768,"thread_start_us":125,"threads_started":1}
I20260809 04:40:39.681020 14637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.680595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.193:44319)
0809 04:40:39.680865 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.680924 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":105,"threads_started":1}
I20260809 04:40:39.681941 14192 heartbeater.cc:503] Master 127.5.222.253:44613 was elected leader, sending a full tablet report...
I20260809 04:40:39.681969 14354 heartbeater.cc:503] Master 127.5.222.253:44613 was elected leader, sending a full tablet report...
I20260809 04:40:39.682037 14637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.681749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.193:53541)
0809 04:40:39.681913 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.681966 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 04:40:39.682396 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.682049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.194:48555)
0809 04:40:39.682277 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.682333 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260809 04:40:39.682718 14635 tablet_bootstrap.cc:492] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6: Bootstrap starting.
I20260809 04:40:39.683758 14635 tablet_bootstrap.cc:654] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:39.684873 14635 tablet_bootstrap.cc:492] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6: No bootstrap required, opened a new log
I20260809 04:40:39.684950 14635 ts_tablet_manager.cc:1397] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:39.685341 14635 raft_consensus.cc:348] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } }
I20260809 04:40:39.685449 14635 raft_consensus.cc:374] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:39.685473 14635 raft_consensus.cc:729] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a56bf57806d4996bf4de69813ea10d6, State: Initialized, Role: FOLLOWER
I20260809 04:40:39.685608 14635 consensus_queue.cc:260] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } }
I20260809 04:40:39.685951 14635 ts_tablet_manager.cc:1428] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:39.686719 14641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.686247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.195:36093)
0809 04:40:39.686554 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.686616 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":136,"threads_started":1}
I20260809 04:40:39.687543 14641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.687180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.195:38213)
0809 04:40:39.687404 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.687480 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260809 04:40:39.754495 14640 raft_consensus.cc:482] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:39.754639 14640 raft_consensus.cc:504] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } }
I20260809 04:40:39.755825 14640 leader_election.cc:288] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b660677db014b769e872ce5abb04084 (127.5.222.194:43927), 8750546d7c214df3902dfe9a782667c0 (127.5.222.193:38087)
I20260809 04:40:39.760210 14641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.756074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:38087 (local address 127.5.222.195:38761)
0809 04:40:39.756216 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.756235 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:39.756349 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:39.756631 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:39.756634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:39.756652 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:39.756783 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.756788 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.757800 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.757803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.759053 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.759064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.759829 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.759831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.759896 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.759916 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:40:39.759973 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 04:40:39.760067 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 04:40:39.760221 14624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.756196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:38761 (local address 127.5.222.193:38087)
0809 04:40:39.756328 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:39.756332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:39.756348 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:40:39.756478 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:39.756483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:39.756528 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:39.756616 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:39.756926 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.757669 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.759166 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.759714 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.759834 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.759885 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:39.760022 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 04:40:39.760097 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:40:39.760628 14623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.756180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:34235 (local address 127.5.222.194:43927)
0809 04:40:39.756320 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:39.756324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:39.756703 (+ 379us) server_negotiation.cc:407] Connection header received
0809 04:40:39.756866 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:39.756869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:39.756936 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:39.757021 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:39.757329 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.757972 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.759361 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.759946 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.760081 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.760123 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:39.760399 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 04:40:39.760495 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 04:40:39.760767 14141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be069dd1223e47b2990bcf9e93f46cd2" candidate_uuid: "1a56bf57806d4996bf4de69813ea10d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8750546d7c214df3902dfe9a782667c0" is_pre_election: true
I20260809 04:40:39.761013 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.756074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43927 (local address 127.5.222.195:34235)
0809 04:40:39.756595 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.756606 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:39.756750 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:39.757036 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:39.757038 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:39.757054 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:39.757193 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.757199 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.758073 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.758076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.759257 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:39.759264 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.760107 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.760111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:39.760205 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.760265 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 04:40:39.760780 (+ 515us) client_negotiation.cc:240] Negotiation successful
0809 04:40:39.760859 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":206,"thread_start_us":157,"threads_started":1}
I20260809 04:40:39.761014 14141 raft_consensus.cc:2454] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a56bf57806d4996bf4de69813ea10d6 in term 0.
I20260809 04:40:39.761471 14298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be069dd1223e47b2990bcf9e93f46cd2" candidate_uuid: "1a56bf57806d4996bf4de69813ea10d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b660677db014b769e872ce5abb04084" is_pre_election: true
I20260809 04:40:39.761420 14381 leader_election.cc:302] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a56bf57806d4996bf4de69813ea10d6, 8750546d7c214df3902dfe9a782667c0; no voters:
I20260809 04:40:39.761620 14298 raft_consensus.cc:2454] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a56bf57806d4996bf4de69813ea10d6 in term 0.
I20260809 04:40:39.761708 14640 raft_consensus.cc:2781] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:39.761770 14640 raft_consensus.cc:482] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:39.761824 14640 raft_consensus.cc:3037] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:39.763065 14640 raft_consensus.cc:504] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } }
I20260809 04:40:39.763391 14640 leader_election.cc:288] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [CANDIDATE]: Term 1 election: Requested vote from peers 0b660677db014b769e872ce5abb04084 (127.5.222.194:43927), 8750546d7c214df3902dfe9a782667c0 (127.5.222.193:38087)
I20260809 04:40:39.763700 14298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be069dd1223e47b2990bcf9e93f46cd2" candidate_uuid: "1a56bf57806d4996bf4de69813ea10d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b660677db014b769e872ce5abb04084"
I20260809 04:40:39.763782 14141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be069dd1223e47b2990bcf9e93f46cd2" candidate_uuid: "1a56bf57806d4996bf4de69813ea10d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8750546d7c214df3902dfe9a782667c0"
I20260809 04:40:39.763835 14298 raft_consensus.cc:3037] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:39.763926 14141 raft_consensus.cc:3037] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:39.764775 14298 raft_consensus.cc:2454] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a56bf57806d4996bf4de69813ea10d6 in term 1.
I20260809 04:40:39.765030 14141 raft_consensus.cc:2454] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a56bf57806d4996bf4de69813ea10d6 in term 1.
I20260809 04:40:39.765069 14382 leader_election.cc:302] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b660677db014b769e872ce5abb04084, 1a56bf57806d4996bf4de69813ea10d6; no voters:
I20260809 04:40:39.765309 14640 raft_consensus.cc:2781] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:39.765563 14640 raft_consensus.cc:686] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 1 LEADER]: Becoming Leader. State: Replica: 1a56bf57806d4996bf4de69813ea10d6, State: Running, Role: LEADER
I20260809 04:40:39.765770 14513 heartbeater.cc:503] Master 127.5.222.253:44613 was elected leader, sending a full tablet report...
I20260809 04:40:39.765946 14640 consensus_queue.cc:237] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } } peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } }
I20260809 04:40:39.766227 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.765995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.195:56301)
0809 04:40:39.766104 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.766176 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 04:40:39.768231 13782 catalog_manager.cc:5495] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a56bf57806d4996bf4de69813ea10d6 (127.5.222.195). New cstate: current_term: 1 leader_uuid: "1a56bf57806d4996bf4de69813ea10d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a56bf57806d4996bf4de69813ea10d6" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38385 } health_report { overall_health: HEALTHY } } }
I20260809 04:40:39.794343 14626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.790715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34160 (local address 127.5.222.195:38385)
0809 04:40:39.790875 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:39.790880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:39.790894 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:40:39.791067 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:39.791071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:39.791134 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:39.791254 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:39.791501 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.792011 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.792893 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:39.793126 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:39.793267 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:39.793522 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:39.793941 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:39.794069 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:39.794130 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 04:40:39.794185 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:40:39.797178 14298 raft_consensus.cc:1261] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084 [term 1 FOLLOWER]: Refusing update from remote peer 1a56bf57806d4996bf4de69813ea10d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:39.797397 14141 raft_consensus.cc:1261] T be069dd1223e47b2990bcf9e93f46cd2 P 8750546d7c214df3902dfe9a782667c0 [term 1 FOLLOWER]: Refusing update from remote peer 1a56bf57806d4996bf4de69813ea10d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:39.797717 14643 consensus_queue.cc:1035] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b660677db014b769e872ce5abb04084" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:39.797996 14640 consensus_queue.cc:1035] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8750546d7c214df3902dfe9a782667c0" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:39.799516 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.799225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.194:60283)
0809 04:40:39.799364 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.799434 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 04:40:39.802079 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.801781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.195:57967)
0809 04:40:39.801946 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.802005 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 04:40:39.802731 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.802446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.194:53407)
0809 04:40:39.802611 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.802662 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 04:40:39.802814 14637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.802557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.193:43211)
0809 04:40:39.802696 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.802745 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 04:40:39.804296 14647 mvcc.cc:205] Tried to move back new op lower bound from 7316481801403596800 to 7316481801284210688. Current Snapshot: MvccSnapshot[applied={T|T < 7316481801403596800}]
I20260809 04:40:39.806529 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.806232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.195:43025)
0809 04:40:39.806396 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.806462 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 04:40:39.810359 14637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.807404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43161 (local address 127.5.222.193:58913)
0809 04:40:39.810229 (+ 2825us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.810286 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:43161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2755}
I20260809 04:40:39.810899 13782 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:39.811069 13782 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:39.814941 13782 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d405a7019604bb88d8db6f3656676c4: Sending DeleteTablet for 3 replicas of tablet be069dd1223e47b2990bcf9e93f46cd2
I20260809 04:40:39.815605 14270 tablet_service.cc:1514] Processing DeleteTablet for tablet be069dd1223e47b2990bcf9e93f46cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:39 UTC) from {username='oryx', principal='oryx/127.5.222.253@KRBTEST.COM'} at 127.0.0.1:44362
I20260809 04:40:39.815701 14433 tablet_service.cc:1514] Processing DeleteTablet for tablet be069dd1223e47b2990bcf9e93f46cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:39 UTC) from {username='oryx', principal='oryx/127.5.222.253@KRBTEST.COM'} at 127.0.0.1:34156
I20260809 04:40:39.816133 14116 tablet_service.cc:1514] Processing DeleteTablet for tablet be069dd1223e47b2990bcf9e93f46cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:39 UTC) from {username='oryx', principal='oryx/127.5.222.253@KRBTEST.COM'} at 127.0.0.1:38868
I20260809 04:40:39.816360 14659 tablet_replica.cc:330] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6: stopping tablet replica
I20260809 04:40:39.816480 14659 raft_consensus.cc:2229] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:39.816488 14658 tablet_replica.cc:330] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084: stopping tablet replica
I20260809 04:40:39.816501 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 13944
I20260809 04:40:39.818141 14659 raft_consensus.cc:2258] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:39.819445 14659 ts_tablet_manager.cc:1905] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:39.823348 14659 ts_tablet_manager.cc:1918] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:39.823441 14659 log.cc:1196] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/be069dd1223e47b2990bcf9e93f46cd2
I20260809 04:40:39.823803 14659 ts_tablet_manager.cc:1939] T be069dd1223e47b2990bcf9e93f46cd2 P 1a56bf57806d4996bf4de69813ea10d6: Deleting consensus metadata
I20260809 04:40:39.816594 14658 raft_consensus.cc:2229] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:39.826656 14658 raft_consensus.cc:2258] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:39.828181 14658 ts_tablet_manager.cc:1905] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:39.830156 13762 catalog_manager.cc:4841] TS 1a56bf57806d4996bf4de69813ea10d6 (127.5.222.195:38385): tablet be069dd1223e47b2990bcf9e93f46cd2 (table test-table [id=1222534f47af43fe9f2fdfb1d371481a]) successfully deleted
I20260809 04:40:39.830754 14658 ts_tablet_manager.cc:1918] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:39.830830 14658 log.cc:1196] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/be069dd1223e47b2990bcf9e93f46cd2
I20260809 04:40:39.831089 14658 ts_tablet_manager.cc:1939] T be069dd1223e47b2990bcf9e93f46cd2 P 0b660677db014b769e872ce5abb04084: Deleting consensus metadata
I20260809 04:40:39.831787 13766 catalog_manager.cc:4841] TS 0b660677db014b769e872ce5abb04084 (127.5.222.194:43927): tablet be069dd1223e47b2990bcf9e93f46cd2 (table test-table [id=1222534f47af43fe9f2fdfb1d371481a]) successfully deleted
I20260809 04:40:39.837786 14625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:39.837511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:38087 (local address 127.0.0.1:38870)
0809 04:40:39.837650 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:39.837706 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 04:40:39.837991 13764 catalog_manager.cc:4584] TS 8750546d7c214df3902dfe9a782667c0 (127.5.222.193:38087): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be069dd1223e47b2990bcf9e93f46cd2: Network error: Client connection negotiation failed: client connection to 127.5.222.193:38087: connect: Connection refused (error 111)
I20260809 04:40:39.838192 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 14200
I20260809 04:40:39.848058 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 14359
I20260809 04:40:39.856186 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 13623
I20260809 04:40:39.864512 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 13742
2026-08-09T04:40:39Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5585 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:39 dist-test-slave-vxj2 krb5kdc[14663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:39 dist-test-slave-vxj2 krb5kdc[14663](info): set up 2 sockets
Aug 09 04:40:39 dist-test-slave-vxj2 krb5kdc[14663](info): commencing operation
krb5kdc: starting...
W20260809 04:40:40.988600 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:41 dist-test-slave-vxj2 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:41Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.254.keytab.
Entry for principal oryx/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.254.keytab.
I20260809 04:40:41.114470 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:40455
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.254.keytab
--principal=oryx/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:40455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:41.224498 14677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:41.224766 14677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:41.224807 14677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:41.228968 14677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:41.229031 14677 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 04:40:41.229048 14677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:41.229065 14677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:41.229082 14677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:41.229125 14677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:41.234496 14677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:40455
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.254.keytab
--principal=oryx/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:40455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:41.235525 14677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:41.236521 14677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:41.244980 14682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:41.245067 14683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:41.245021 14685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:41.245209 14677 server_base.cc:1034] running on GCE node
I20260809 04:40:41.245850 14677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:41.246963 14677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:41.248169 14677 hybrid_clock.cc:648] HybridClock initialized: now 1786250441248150 us; error 37 us; skew 500 ppm
Aug 09 04:40:41 dist-test-slave-vxj2 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250441, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:41.251083 14677 init.cc:377] Logged in from keytab as oryx/127.5.222.254@KRBTEST.COM (short username oryx)
I20260809 04:40:41.251943 14677 webserver.cc:460] Webserver started at http://127.5.222.254:42473/ using document root <none> and password file <none>
I20260809 04:40:41.252509 14677 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:41.252601 14677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:41.252853 14677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:41.254976 14677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "5c000cfc0a89429eb13779a5ec388655"
format_stamp: "Formatted at 2026-08-09 04:40:41 on dist-test-slave-vxj2"
server_key: "ec1b5c879b73e4583474c4bd38736396"
server_key_iv: "9414fd76f988503ff478e62656fbd54c"
server_key_version: "encryptionkey@0"
I20260809 04:40:41.255553 14677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "5c000cfc0a89429eb13779a5ec388655"
format_stamp: "Formatted at 2026-08-09 04:40:41 on dist-test-slave-vxj2"
server_key: "ec1b5c879b73e4583474c4bd38736396"
server_key_iv: "9414fd76f988503ff478e62656fbd54c"
server_key_version: "encryptionkey@0"
I20260809 04:40:41.259326 14677 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:40:41.261731 14692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:41.262930 14677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:41.263089 14677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "5c000cfc0a89429eb13779a5ec388655"
format_stamp: "Formatted at 2026-08-09 04:40:41 on dist-test-slave-vxj2"
server_key: "ec1b5c879b73e4583474c4bd38736396"
server_key_iv: "9414fd76f988503ff478e62656fbd54c"
server_key_version: "encryptionkey@0"
I20260809 04:40:41.263231 14677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:41.283317 14677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:41.286401 14677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:41.286643 14677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:41.295372 14677 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:40455
I20260809 04:40:41.295380 14763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:40455 every 8 connection(s)
I20260809 04:40:41.296372 14677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:41.299072 14764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:41.300838 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 14677
I20260809 04:40:41.300978 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:41.301324 6011 external_mini_cluster.cc:1428] Setting key c63176adb159ce721e5eee97125949bc
I20260809 04:40:41.305822 14764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655: Bootstrap starting.
I20260809 04:40:41.308298 14764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:41.309221 14764 log.cc:824] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:40:41 dist-test-slave-vxj2 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:41.311357 14764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655: No bootstrap required, opened a new log
I20260809 04:40:41.314709 14764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c000cfc0a89429eb13779a5ec388655" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40455 } }
I20260809 04:40:41.314968 14764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:41.315030 14764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c000cfc0a89429eb13779a5ec388655, State: Initialized, Role: FOLLOWER
I20260809 04:40:41.315495 14764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c000cfc0a89429eb13779a5ec388655" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40455 } }
I20260809 04:40:41.315626 14764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:41.315711 14764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:41.315805 14764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:41.316809 14764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c000cfc0a89429eb13779a5ec388655" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40455 } }
I20260809 04:40:41.317114 14764 leader_election.cc:302] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c000cfc0a89429eb13779a5ec388655; no voters:
I20260809 04:40:41.317462 14764 leader_election.cc:288] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:41.317539 14769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:41.317598 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:41.302813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57042 (local address 127.5.222.254:40455)
0809 04:40:41.303230 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:41.303257 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:41.303294 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:40:41.304016 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:41.304034 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:41.304365 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:41.304648 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:41.305583 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.306809 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.307975 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.308519 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.308645 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:41.311437 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:41.311442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:41.311476 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:41.311508 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:41.314115 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:41.314590 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:41.314596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:41.314602 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:41.314720 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:41.315005 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:41.315009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:41.315011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:41.315561 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:41.315782 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:41.316261 (+ 479us) server_negotiation.cc:299] Negotiation successful
0809 04:40:41.316596 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":137,"threads_started":1}
I20260809 04:40:41.317725 14769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [term 1 LEADER]: Becoming Leader. State: Replica: 5c000cfc0a89429eb13779a5ec388655, State: Running, Role: LEADER
I20260809 04:40:41.318003 14769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c000cfc0a89429eb13779a5ec388655" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40455 } }
I20260809 04:40:41.318714 14764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:41.319708 14772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c000cfc0a89429eb13779a5ec388655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c000cfc0a89429eb13779a5ec388655" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40455 } } }
I20260809 04:40:41.319921 14772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:41.320267 14770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c000cfc0a89429eb13779a5ec388655. Latest consensus state: current_term: 1 leader_uuid: "5c000cfc0a89429eb13779a5ec388655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c000cfc0a89429eb13779a5ec388655" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40455 } } }
I20260809 04:40:41.320370 14770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655 [sys.catalog]: This master's current role is: LEADER
W20260809 04:40:41.323268 14790 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:40:41.323330 14790 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:40:41.323422 14781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:41.324283 14781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:41.329734 14781 catalog_manager.cc:1347] Generated new cluster ID: 3c73a44440ce4e2da4eb7a1a49f9cd35
I20260809 04:40:41.329800 14781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:41.340602 14781 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:41.341981 14781 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:41.346840 14781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655: Generated new TSK 0
I20260809 04:40:41.347507 14781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab.
Entry for principal oryx/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab.
I20260809 04:40:41.380415 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40455
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab
--principal=oryx/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:41.496728 14795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:41.496981 14795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:41.497025 14795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:41.501078 14795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:41.501140 14795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:41.501224 14795 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:40:41.501294 14795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:41.506486 14795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab
--principal=oryx/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.222.254:40455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:41.507472 14795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:41.508540 14795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:41.518107 14801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:41.518113 14800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:41.518177 14803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:41.518414 14795 server_base.cc:1034] running on GCE node
I20260809 04:40:41.518792 14795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:41.519510 14795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:41.520682 14795 hybrid_clock.cc:648] HybridClock initialized: now 1786250441520667 us; error 35 us; skew 500 ppm
Aug 09 04:40:41 dist-test-slave-vxj2 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250441, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:41.523720 14795 init.cc:377] Logged in from keytab as oryx/127.5.222.193@KRBTEST.COM (short username oryx)
I20260809 04:40:41.524580 14795 webserver.cc:460] Webserver started at http://127.5.222.193:46121/ using document root <none> and password file <none>
I20260809 04:40:41.525133 14795 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:41.525260 14795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:41.525491 14795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:41.527500 14795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "173a296a031a4a8cb49dfd26f3bd0ca7"
format_stamp: "Formatted at 2026-08-09 04:40:41 on dist-test-slave-vxj2"
server_key: "7b0f76067438836c2954ee6c6beb8634"
server_key_iv: "696e832215c3a6b261c421064bda653f"
server_key_version: "encryptionkey@0"
I20260809 04:40:41.528069 14795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "173a296a031a4a8cb49dfd26f3bd0ca7"
format_stamp: "Formatted at 2026-08-09 04:40:41 on dist-test-slave-vxj2"
server_key: "7b0f76067438836c2954ee6c6beb8634"
server_key_iv: "696e832215c3a6b261c421064bda653f"
server_key_version: "encryptionkey@0"
I20260809 04:40:41.531725 14795 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:40:41.534112 14811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:41.535274 14795 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:41.535403 14795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "173a296a031a4a8cb49dfd26f3bd0ca7"
format_stamp: "Formatted at 2026-08-09 04:40:41 on dist-test-slave-vxj2"
server_key: "7b0f76067438836c2954ee6c6beb8634"
server_key_iv: "696e832215c3a6b261c421064bda653f"
server_key_version: "encryptionkey@0"
I20260809 04:40:41.535506 14795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:41.553004 14795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:41.555759 14795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:41.555958 14795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:41.557441 14795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:41.557492 14795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:41.557559 14795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:41.557593 14795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:40:41 dist-test-slave-vxj2 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250441, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:41.568975 14795 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:42341
I20260809 04:40:41.569054 14945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:42341 every 8 connection(s)
I20260809 04:40:41.569954 14795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:41.571652 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:41.558280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:38427 (local address 127.5.222.254:40455)
0809 04:40:41.558413 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:41.558418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:41.559471 (+ 1053us) server_negotiation.cc:407] Connection header received
0809 04:40:41.560449 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:41.560455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:41.560527 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:41.560640 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:41.561995 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.562526 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.563836 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.564077 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.564186 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:41.567887 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:41.567891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:41.567894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:41.567921 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:41.569519 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:41.570076 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:41.570079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:41.570080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:41.570158 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:41.570562 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:41.570566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:41.570568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:41.570797 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:41.570878 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:41.571384 (+ 506us) server_negotiation.cc:299] Negotiation successful
0809 04:40:41.571500 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:40:41.572363 14821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:41.558841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40455 (local address 127.5.222.193:38427)
0809 04:40:41.559313 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:41.559355 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:41.560182 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:41.560785 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:41.560792 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:41.561232 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:41.561821 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:41.561836 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.562659 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.562664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:41.563702 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:41.563712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.564239 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.564243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:41.564403 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:41.564991 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:41.565018 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:41.567708 (+ 2690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:41.569654 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:41.569663 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:41.569676 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:41.569930 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:41.570261 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:41.570265 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:41.570267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:41.570443 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:41.570904 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:41.570909 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:41.571156 (+ 247us) client_negotiation.cc:769] Sending connection context
0809 04:40:41.571318 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 04:40:41.571543 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":137,"threads_started":1}
I20260809 04:40:41.573755 14946 heartbeater.cc:347] Connected to a master server at 127.5.222.254:40455
I20260809 04:40:41.573953 14946 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:41.574441 14946 heartbeater.cc:511] Master 127.5.222.254:40455 requested a full tablet report, sending...
I20260809 04:40:41.576217 14710 ts_manager.cc:194] Registered new tserver with Master: 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341)
I20260809 04:40:41.577438 14710 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:38427
I20260809 04:40:41.579560 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 14795
I20260809 04:40:41.579701 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:41.580063 6011 external_mini_cluster.cc:1428] Setting key 51255c2c5e12a946037ec44641c1ac1e
I20260809 04:40:41.583506 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:41.575919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:42813 (local address 127.5.222.254:40455)
0809 04:40:41.576046 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:41.576049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:41.576153 (+ 104us) server_negotiation.cc:407] Connection header received
0809 04:40:41.576273 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:41.576275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:41.576308 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:41.576406 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:41.576863 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.577420 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.578370 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.578586 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.578693 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:41.579946 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:41.579949 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:41.579951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:41.579972 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:41.581753 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:41.582277 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:41.582280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:41.582281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:41.582362 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:41.582679 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:41.582682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:41.582683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:41.582844 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:41.582924 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:41.583303 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 04:40:41.583400 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:40:41.583675 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:41.575778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40455 (local address 127.5.222.193:42813)
0809 04:40:41.576083 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:41.576099 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:41.576194 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:41.576468 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:41.576471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:41.576655 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:41.576746 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:41.576751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.577544 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.577548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:41.578250 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:41.578262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.578707 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.578711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:41.578811 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:41.579342 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:41.579359 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:41.579813 (+ 454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:41.581868 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:41.581871 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:41.581873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:41.582147 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:41.582466 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:41.582472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:41.582477 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:41.582584 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:41.582947 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:41.582953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:41.583067 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 04:40:41.583226 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 04:40:41.583397 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":131,"threads_started":1}
WARNING: no policy specified for oryx/127.5.222.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab.
Entry for principal oryx/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab.
I20260809 04:40:41.608673 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40455
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab
--principal=oryx/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:41.718952 14958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:41.719241 14958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:41.719290 14958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:41.723496 14958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:41.723547 14958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:41.723587 14958 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:40:41.723619 14958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:41.728782 14958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab
--principal=oryx/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.222.254:40455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:41.729807 14958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:41.730773 14958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:41.740437 14965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:41.740398 14963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:41.740506 14967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:41.740787 14958 server_base.cc:1034] running on GCE node
I20260809 04:40:41.741254 14958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:41.741921 14958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:41.743108 14958 hybrid_clock.cc:648] HybridClock initialized: now 1786250441743098 us; error 30 us; skew 500 ppm
Aug 09 04:40:41 dist-test-slave-vxj2 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250441, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:41.745922 14958 init.cc:377] Logged in from keytab as oryx/127.5.222.194@KRBTEST.COM (short username oryx)
I20260809 04:40:41.746750 14958 webserver.cc:460] Webserver started at http://127.5.222.194:45225/ using document root <none> and password file <none>
I20260809 04:40:41.747290 14958 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:41.747380 14958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:41.747610 14958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:41.749675 14958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "be696c8e15ee488e9915ed6330f3f614"
format_stamp: "Formatted at 2026-08-09 04:40:41 on dist-test-slave-vxj2"
server_key: "bf334d93285f51eb5097d5246f2e4752"
server_key_iv: "e8891adc62bddbcf0c029e40ebd83e7a"
server_key_version: "encryptionkey@0"
I20260809 04:40:41.750257 14958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "be696c8e15ee488e9915ed6330f3f614"
format_stamp: "Formatted at 2026-08-09 04:40:41 on dist-test-slave-vxj2"
server_key: "bf334d93285f51eb5097d5246f2e4752"
server_key_iv: "e8891adc62bddbcf0c029e40ebd83e7a"
server_key_version: "encryptionkey@0"
I20260809 04:40:41.753857 14958 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:40:41.756122 14974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:41.757359 14958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 04:40:41.757484 14958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "be696c8e15ee488e9915ed6330f3f614"
format_stamp: "Formatted at 2026-08-09 04:40:41 on dist-test-slave-vxj2"
server_key: "bf334d93285f51eb5097d5246f2e4752"
server_key_iv: "e8891adc62bddbcf0c029e40ebd83e7a"
server_key_version: "encryptionkey@0"
I20260809 04:40:41.757589 14958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:41.776079 14958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:41.779291 14958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:41.779497 14958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:41.780894 14958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:41.780966 14958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:41.781040 14958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:41.781085 14958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:40:41 dist-test-slave-vxj2 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250441, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:41.792380 14958 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:32899
I20260809 04:40:41.792536 15101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:32899 every 8 connection(s)
I20260809 04:40:41.793385 14958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:41.795084 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:41.781782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:38983 (local address 127.5.222.254:40455)
0809 04:40:41.781911 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:41.781916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:41.782923 (+ 1007us) server_negotiation.cc:407] Connection header received
0809 04:40:41.784074 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:41.784079 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:41.784144 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:41.784228 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:41.785985 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.786939 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.788237 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.788600 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.788746 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:41.791298 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:41.791302 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:41.791305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:41.791334 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:41.792955 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:41.793530 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:41.793536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:41.793538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:41.793624 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:41.794045 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:41.794048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:41.794050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:41.794260 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:41.794336 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:41.794762 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 04:40:41.794901 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:40:41.795132 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 14958
I20260809 04:40:41.795244 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:41.795591 6011 external_mini_cluster.cc:1428] Setting key 951967b902757bc17abdff0e45046d78
I20260809 04:40:41.795815 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:41.782285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40455 (local address 127.5.222.194:38983)
0809 04:40:41.782769 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:41.782809 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:41.783819 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:41.784734 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:41.784744 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:41.785243 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:41.785835 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:41.785859 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.787084 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.787088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:41.788092 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:41.788101 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.788842 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.788849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:41.789064 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:41.789678 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:41.789693 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:41.791133 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:41.793083 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:41.793088 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:41.793102 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:41.793423 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:41.793730 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:41.793733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:41.793735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:41.793926 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:41.794352 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:41.794356 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:41.794558 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 04:40:41.794698 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 04:40:41.794925 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":135,"threads_started":1}
I20260809 04:40:41.798720 15102 heartbeater.cc:347] Connected to a master server at 127.5.222.254:40455
I20260809 04:40:41.798866 15102 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:41.799322 15102 heartbeater.cc:511] Master 127.5.222.254:40455 requested a full tablet report, sending...
I20260809 04:40:41.800305 14710 ts_manager.cc:194] Registered new tserver with Master: be696c8e15ee488e9915ed6330f3f614 (127.5.222.194:32899)
I20260809 04:40:41.801085 14710 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:38983
WARNING: no policy specified for oryx/127.5.222.195@KRBTEST.COM; defaulting to no policy
I20260809 04:40:41.807579 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:41.798988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:39113 (local address 127.5.222.254:40455)
0809 04:40:41.799114 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:41.799118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:41.799268 (+ 150us) server_negotiation.cc:407] Connection header received
0809 04:40:41.799376 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:41.799379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:41.799422 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:41.799506 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:41.800067 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.800644 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.803048 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.803278 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.803392 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:41.804466 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:41.804469 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:41.804472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:41.804494 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:41.805901 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:41.806311 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:41.806315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:41.806317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:41.806398 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:41.806724 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:41.806727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:41.806729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:41.806955 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:41.807070 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:41.807322 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 04:40:41.807429 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:40:41.807596 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:41.798934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40455 (local address 127.5.222.194:39113)
0809 04:40:41.799168 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:41.799186 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:41.799297 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:41.799540 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:41.799544 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:41.799769 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:41.799902 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:41.799908 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.800826 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.800833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:41.801735 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:41.801744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.803409 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.803412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:41.803510 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:41.804006 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:41.804020 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:41.804317 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:41.806012 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:41.806016 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:41.806019 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:41.806211 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:41.806496 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:41.806500 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:41.806502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:41.806588 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:41.807059 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:41.807062 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:41.807158 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 04:40:41.807267 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 04:40:41.807415 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab.
Entry for principal oryx/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab.
I20260809 04:40:41.824527 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40455
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab
--principal=oryx/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:41.936899 15115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:41.937143 15115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:41.937264 15115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:41.941414 15115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:41.941470 15115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:41.941509 15115 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:40:41.941576 15115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:41.946682 15115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab
--principal=oryx/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.222.254:40455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:41.947685 15115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:41.948693 15115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:41.958190 15123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:41.958169 15125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:41.958168 15120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:41.958235 15115 server_base.cc:1034] running on GCE node
I20260809 04:40:41.958813 15115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:41.959400 15115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:41.960570 15115 hybrid_clock.cc:648] HybridClock initialized: now 1786250441960543 us; error 47 us; skew 500 ppm
Aug 09 04:40:41 dist-test-slave-vxj2 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250441, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:41.963450 15115 init.cc:377] Logged in from keytab as oryx/127.5.222.195@KRBTEST.COM (short username oryx)
I20260809 04:40:41.964272 15115 webserver.cc:460] Webserver started at http://127.5.222.195:40291/ using document root <none> and password file <none>
I20260809 04:40:41.964807 15115 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:41.964893 15115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:41.965118 15115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:41.967196 15115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "66897116c1df45759027f5fbf9c200f7"
format_stamp: "Formatted at 2026-08-09 04:40:41 on dist-test-slave-vxj2"
server_key: "12cdba993a0d4fdab6aa45c71df16f70"
server_key_iv: "e1d9305d1c4c8e6a37a450469fb186d1"
server_key_version: "encryptionkey@0"
I20260809 04:40:41.967736 15115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "66897116c1df45759027f5fbf9c200f7"
format_stamp: "Formatted at 2026-08-09 04:40:41 on dist-test-slave-vxj2"
server_key: "12cdba993a0d4fdab6aa45c71df16f70"
server_key_iv: "e1d9305d1c4c8e6a37a450469fb186d1"
server_key_version: "encryptionkey@0"
I20260809 04:40:41.971341 15115 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 04:40:41.973798 15132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:41.975131 15115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:41.975273 15115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "66897116c1df45759027f5fbf9c200f7"
format_stamp: "Formatted at 2026-08-09 04:40:41 on dist-test-slave-vxj2"
server_key: "12cdba993a0d4fdab6aa45c71df16f70"
server_key_iv: "e1d9305d1c4c8e6a37a450469fb186d1"
server_key_version: "encryptionkey@0"
I20260809 04:40:41.975381 15115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:41.987221 15115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:41.990479 15115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:41.990680 15115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:41.992110 15115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:41.992185 15115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:41.992256 15115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:41.992298 15115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:42.002753 15115 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:42803
I20260809 04:40:42.002801 15265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:42803 every 8 connection(s)
Aug 09 04:40:42 dist-test-slave-vxj2 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250441, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:42.003787 15115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:42.007488 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:41.993120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:42129 (local address 127.5.222.254:40455)
0809 04:40:41.993298 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:41.993302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:41.994326 (+ 1024us) server_negotiation.cc:407] Connection header received
0809 04:40:41.995695 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:41.995702 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:41.995779 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:41.995884 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:41.997372 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.998057 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.999955 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.000330 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.000476 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.003654 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:42.003658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:42.003661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:42.003697 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:42.005587 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:42.006091 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:42.006094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:42.006095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:42.006175 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:42.006488 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:42.006490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:42.006491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:42.006675 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:42.006750 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.007256 (+ 506us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.007366 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 04:40:42.008374 15141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:41.993681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40455 (local address 127.5.222.195:42129)
0809 04:40:41.994170 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:41.994211 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:41.995376 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:41.996117 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:41.996128 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:41.996510 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:41.997148 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:41.997219 (+ 71us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:41.998186 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:41.998190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:41.999783 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:41.999798 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.000504 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.000507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.000651 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.001432 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:42.001467 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:42.003448 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:42.005703 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:42.005708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:42.005719 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:42.005983 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:42.006260 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:42.006262 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:42.006264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:42.006394 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:42.006783 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:42.006805 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:42.007038 (+ 233us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.007188 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.007459 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":135,"threads_started":1}
I20260809 04:40:42.009697 15266 heartbeater.cc:347] Connected to a master server at 127.5.222.254:40455
I20260809 04:40:42.009861 15266 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:42.010375 15266 heartbeater.cc:511] Master 127.5.222.254:40455 requested a full tablet report, sending...
I20260809 04:40:42.011238 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 15115
I20260809 04:40:42.011402 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:40:42.011521 14710 ts_manager.cc:194] Registered new tserver with Master: 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803)
I20260809 04:40:42.011755 6011 external_mini_cluster.cc:1428] Setting key 38e790b3102765f09c806fed37db455a
I20260809 04:40:42.012288 14710 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:42129
I20260809 04:40:42.014034 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:40:42.019944 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.012159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40455 (local address 127.5.222.195:57645)
0809 04:40:42.012532 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.012548 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.012645 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.012950 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.012953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.013215 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:42.013343 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.013350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.014243 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.014246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.014794 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.014802 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.015256 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.015260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.015351 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.015892 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:42.015922 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:42.016409 (+ 487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:42.018246 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:42.018253 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:42.018258 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:42.018564 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:42.018891 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:42.018896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:42.018900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:42.019016 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:42.019349 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:42.019352 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:42.019462 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.019577 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.019723 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":2,"thread_start_us":118,"threads_started":1}
I20260809 04:40:42.020256 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.012212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:57645 (local address 127.5.222.254:40455)
0809 04:40:42.012320 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.012324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.012625 (+ 301us) server_negotiation.cc:407] Connection header received
0809 04:40:42.012773 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.012777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.012827 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.012923 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:42.013463 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.014125 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.014914 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.015144 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.015252 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.016554 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:42.016557 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:42.016559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:42.016582 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:42.018129 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:42.018668 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:42.018671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:42.018672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:42.018796 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:42.019115 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:42.019117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:42.019118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:42.019273 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:42.019346 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.019672 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.019848 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 04:40:42 dist-test-slave-vxj2 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250442, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:40:42 dist-test-slave-vxj2 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:42 dist-test-slave-vxj2 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:40:42.039680 14767 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:42.033288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57052 (local address 127.5.222.254:40455)
0809 04:40:42.033407 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.033411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.033422 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:40:42.033463 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.033465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.033503 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.033570 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:42.034313 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.034921 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.035778 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.036008 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.036088 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.039515 (+ 3427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57052: BlockingRecv error: recv got EOF from 127.0.0.1:57052 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 04:40:42 dist-test-slave-vxj2 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250442, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:42.049239 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.041173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57054 (local address 127.5.222.254:40455)
0809 04:40:42.041382 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.041386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.041507 (+ 121us) server_negotiation.cc:407] Connection header received
0809 04:40:42.041650 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.041653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.041698 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.041839 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:42.042468 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.042958 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.043755 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.043957 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.044068 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.046323 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:42.046333 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:42.046336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:42.046357 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:42.047569 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:42.047990 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:42.047994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:42.047996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:42.048077 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:42.048359 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:42.048363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:42.048364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:42.048536 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:42.048681 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.048911 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.049034 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260809 04:40:42.052129 14710 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:42.055318 14710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:42.070206 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.063945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56406 (local address 127.5.222.194:32899)
0809 04:40:42.064996 (+ 1051us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.065007 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.065031 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:40:42.065107 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.065112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.065334 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.065834 (+ 500us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:42.065961 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.067071 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.068902 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.069528 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.069654 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.069823 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.069964 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.070054 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":950,"thread_start_us":98,"threads_started":1}
I20260809 04:40:42.070200 15291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.063830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:32899 (local address 127.0.0.1:56406)
0809 04:40:42.064574 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.064621 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.064757 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.065525 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.065531 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.065552 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:42.065680 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.065687 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.067194 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.067197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.068758 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.068769 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.069652 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.069656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.069776 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.069821 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.069934 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.070055 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":117,"spinlock_wait_cycles":1280,"thread_start_us":88,"threads_started":1}
I20260809 04:40:42.070614 15294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.064162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:42341 (local address 127.0.0.1:33172)
0809 04:40:42.065309 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.065323 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.065473 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.066133 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.066135 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.066177 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:42.066253 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.066259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.067609 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.067616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.068770 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.068780 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.069936 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.069940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.070026 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.070052 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.070119 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.070189 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1089,"mutex_wait_us":46,"thread_start_us":85,"threads_started":1}
I20260809 04:40:42.070767 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.064431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33172 (local address 127.5.222.193:42341)
0809 04:40:42.064777 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.064792 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.065404 (+ 612us) server_negotiation.cc:407] Connection header received
0809 04:40:42.065628 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.065635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.065840 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.065966 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:42.066372 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.067497 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.068897 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.069604 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.069712 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.069858 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.070429 (+ 571us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.070526 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":73,"threads_started":1}
I20260809 04:40:42.070784 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.063885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:42803 (local address 127.0.0.1:53336)
0809 04:40:42.064573 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.064620 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.064757 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.065637 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.065640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.065653 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:42.065751 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.065760 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.067048 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.067053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.068758 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.068769 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.070358 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.070363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.070481 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.070502 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.070573 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.070635 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":128,"spinlock_wait_cycles":5888,"thread_start_us":111,"threads_started":1}
I20260809 04:40:42.071866 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.064568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.5.222.195:42803)
0809 04:40:42.065080 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.065091 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.065109 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:40:42.065197 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.065201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.065340 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.065496 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:42.065879 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.066940 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.069276 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.070038 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.070142 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.070286 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.070971 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.071073 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":98,"threads_started":1}
I20260809 04:40:42.073529 15032 tablet_service.cc:1467] Processing CreateTablet for tablet 2323a4be46aa42b59725f42da73ca48b (DEFAULT_TABLE table=test-table [id=3041661474a848e281622f7f593803b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:42.073553 14871 tablet_service.cc:1467] Processing CreateTablet for tablet 2323a4be46aa42b59725f42da73ca48b (DEFAULT_TABLE table=test-table [id=3041661474a848e281622f7f593803b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:42.074687 14871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2323a4be46aa42b59725f42da73ca48b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:42.074687 15032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2323a4be46aa42b59725f42da73ca48b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:42.075423 15184 tablet_service.cc:1467] Processing CreateTablet for tablet 2323a4be46aa42b59725f42da73ca48b (DEFAULT_TABLE table=test-table [id=3041661474a848e281622f7f593803b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:42.076288 15184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2323a4be46aa42b59725f42da73ca48b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:42.082337 15298 tablet_bootstrap.cc:492] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7: Bootstrap starting.
I20260809 04:40:42.083142 15299 tablet_bootstrap.cc:492] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7: Bootstrap starting.
I20260809 04:40:42.084492 15300 tablet_bootstrap.cc:492] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614: Bootstrap starting.
I20260809 04:40:42.085024 15298 tablet_bootstrap.cc:654] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:42.085786 15298 log.cc:824] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:42.086422 15300 tablet_bootstrap.cc:654] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:42.086596 15299 tablet_bootstrap.cc:654] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:42.087270 15300 log.cc:824] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:42.087270 15299 log.cc:824] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:42.087801 15298 tablet_bootstrap.cc:492] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7: No bootstrap required, opened a new log
I20260809 04:40:42.088016 15298 ts_tablet_manager.cc:1397] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:40:42.089115 15299 tablet_bootstrap.cc:492] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7: No bootstrap required, opened a new log
I20260809 04:40:42.089325 15299 ts_tablet_manager.cc:1397] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:40:42.089517 15300 tablet_bootstrap.cc:492] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614: No bootstrap required, opened a new log
I20260809 04:40:42.089694 15300 ts_tablet_manager.cc:1397] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:40:42.091586 15298 raft_consensus.cc:348] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } }
I20260809 04:40:42.091815 15298 raft_consensus.cc:374] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:42.091869 15298 raft_consensus.cc:729] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66897116c1df45759027f5fbf9c200f7, State: Initialized, Role: FOLLOWER
I20260809 04:40:42.092319 15298 consensus_queue.cc:260] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } }
I20260809 04:40:42.092948 15266 heartbeater.cc:503] Master 127.5.222.254:40455 was elected leader, sending a full tablet report...
I20260809 04:40:42.093338 15298 ts_tablet_manager.cc:1428] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 04:40:42.093310 15300 raft_consensus.cc:348] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } }
I20260809 04:40:42.093310 15299 raft_consensus.cc:348] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } }
I20260809 04:40:42.093539 15300 raft_consensus.cc:374] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:42.093539 15299 raft_consensus.cc:374] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:42.093588 15300 raft_consensus.cc:729] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be696c8e15ee488e9915ed6330f3f614, State: Initialized, Role: FOLLOWER
I20260809 04:40:42.093588 15299 raft_consensus.cc:729] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173a296a031a4a8cb49dfd26f3bd0ca7, State: Initialized, Role: FOLLOWER
I20260809 04:40:42.093993 15300 consensus_queue.cc:260] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } }
I20260809 04:40:42.093993 15299 consensus_queue.cc:260] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } }
I20260809 04:40:42.094624 15102 heartbeater.cc:503] Master 127.5.222.254:40455 was elected leader, sending a full tablet report...
I20260809 04:40:42.094873 15300 ts_tablet_manager.cc:1428] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 04:40:42.094873 15299 ts_tablet_manager.cc:1428] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 04:40:42.095165 14946 heartbeater.cc:503] Master 127.5.222.254:40455 was elected leader, sending a full tablet report...
W20260809 04:40:42.254976 15268 tablet.cc:2369] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:40:42.289062 15305 raft_consensus.cc:482] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:42.289392 15305 raft_consensus.cc:504] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } }
I20260809 04:40:42.290650 15305 leader_election.cc:288] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be696c8e15ee488e9915ed6330f3f614 (127.5.222.194:32899), 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341)
W20260809 04:40:42.294561 15103 tablet.cc:2369] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:40:42.294945 15141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.290960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:32899 (local address 127.5.222.195:50247)
0809 04:40:42.291173 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.291190 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.291293 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.291634 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.291636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.291651 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:42.291800 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.291808 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.292654 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.292657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.293777 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.293788 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.294523 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.294526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.294643 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.294669 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.294745 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.294817 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260809 04:40:42.294982 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.291085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:50247 (local address 127.5.222.194:32899)
0809 04:40:42.291234 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.291238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.291256 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:40:42.291411 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.291414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.291477 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.291579 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:42.291927 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.292539 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.293908 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.294398 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.294517 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.294568 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.294782 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.294853 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:40:42.295564 15052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2323a4be46aa42b59725f42da73ca48b" candidate_uuid: "66897116c1df45759027f5fbf9c200f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be696c8e15ee488e9915ed6330f3f614" is_pre_election: true
I20260809 04:40:42.295722 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.291158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:58853 (local address 127.5.222.193:42341)
0809 04:40:42.291287 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.291292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.291497 (+ 205us) server_negotiation.cc:407] Connection header received
0809 04:40:42.291562 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.291568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.291613 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.291694 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:42.292362 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.292975 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.294262 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.294834 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.294945 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.294990 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.295496 (+ 506us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.295576 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:40:42.295843 15052 raft_consensus.cc:2454] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66897116c1df45759027f5fbf9c200f7 in term 0.
I20260809 04:40:42.296214 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.290960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:42341 (local address 127.5.222.195:58853)
0809 04:40:42.291391 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.291405 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.291492 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.291711 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.291714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.291725 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:42.292229 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.292236 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.293094 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.293098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.294133 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.294141 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.295076 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.295080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.295335 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.295370 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.295932 (+ 562us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.296009 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":31,"thread_start_us":120,"threads_started":1}
I20260809 04:40:42.296348 15136 leader_election.cc:302] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66897116c1df45759027f5fbf9c200f7, be696c8e15ee488e9915ed6330f3f614; no voters:
I20260809 04:40:42.296624 15305 raft_consensus.cc:2781] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:42.296729 15305 raft_consensus.cc:482] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:42.296792 15305 raft_consensus.cc:3037] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:42.296869 14895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2323a4be46aa42b59725f42da73ca48b" candidate_uuid: "66897116c1df45759027f5fbf9c200f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" is_pre_election: true
I20260809 04:40:42.297233 14895 raft_consensus.cc:2454] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66897116c1df45759027f5fbf9c200f7 in term 0.
I20260809 04:40:42.298269 15305 raft_consensus.cc:504] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } }
I20260809 04:40:42.298631 15305 leader_election.cc:288] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [CANDIDATE]: Term 1 election: Requested vote from peers be696c8e15ee488e9915ed6330f3f614 (127.5.222.194:32899), 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341)
I20260809 04:40:42.298934 15052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2323a4be46aa42b59725f42da73ca48b" candidate_uuid: "66897116c1df45759027f5fbf9c200f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be696c8e15ee488e9915ed6330f3f614"
I20260809 04:40:42.299001 14895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2323a4be46aa42b59725f42da73ca48b" candidate_uuid: "66897116c1df45759027f5fbf9c200f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7"
I20260809 04:40:42.299089 15052 raft_consensus.cc:3037] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:42.299134 14895 raft_consensus.cc:3037] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:42.300060 15052 raft_consensus.cc:2454] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66897116c1df45759027f5fbf9c200f7 in term 1.
I20260809 04:40:42.300354 15136 leader_election.cc:302] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66897116c1df45759027f5fbf9c200f7, be696c8e15ee488e9915ed6330f3f614; no voters:
I20260809 04:40:42.300383 14895 raft_consensus.cc:2454] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66897116c1df45759027f5fbf9c200f7 in term 1.
I20260809 04:40:42.300534 15305 raft_consensus.cc:2781] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:42.300854 15305 raft_consensus.cc:686] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 1 LEADER]: Becoming Leader. State: Replica: 66897116c1df45759027f5fbf9c200f7, State: Running, Role: LEADER
I20260809 04:40:42.301254 15305 consensus_queue.cc:237] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } }
I20260809 04:40:42.304301 14710 catalog_manager.cc:5495] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66897116c1df45759027f5fbf9c200f7 (127.5.222.195). New cstate: current_term: 1 leader_uuid: "66897116c1df45759027f5fbf9c200f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } health_report { overall_health: UNKNOWN } } }
W20260809 04:40:42.321352 14947 tablet.cc:2369] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:40:42.367079 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.363447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.5.222.195:42803)
0809 04:40:42.363588 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.363593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.363608 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:40:42.363732 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.363735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.363788 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.363901 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:42.364168 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.364683 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.365579 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.365795 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.365904 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.366204 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:42.366628 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:42.366755 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.366847 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.366922 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:40:42.372664 14895 raft_consensus.cc:1261] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Refusing update from remote peer 66897116c1df45759027f5fbf9c200f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:42.372664 15052 raft_consensus.cc:1261] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Refusing update from remote peer 66897116c1df45759027f5fbf9c200f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:42.373345 15305 consensus_queue.cc:1035] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:42.373529 15310 consensus_queue.cc:1035] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:42.383539 15315 mvcc.cc:205] Tried to move back new op lower bound from 7316481811951575040 to 7316481811669106688. Current Snapshot: MvccSnapshot[applied={T|T < 7316481811951575040}]
I20260809 04:40:42.389732 14710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:42.389940 14710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:42.392271 14710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655: Sending DeleteTablet for 3 replicas of tablet 2323a4be46aa42b59725f42da73ca48b
I20260809 04:40:42.393023 15032 tablet_service.cc:1514] Processing DeleteTablet for tablet 2323a4be46aa42b59725f42da73ca48b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:42 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:56406
I20260809 04:40:42.393100 15184 tablet_service.cc:1514] Processing DeleteTablet for tablet 2323a4be46aa42b59725f42da73ca48b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:42 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:53336
I20260809 04:40:42.393131 14871 tablet_service.cc:1514] Processing DeleteTablet for tablet 2323a4be46aa42b59725f42da73ca48b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:42 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:33172
I20260809 04:40:42.393631 15325 tablet_replica.cc:330] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614: stopping tablet replica
I20260809 04:40:42.393942 15326 tablet_replica.cc:330] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7: stopping tablet replica
I20260809 04:40:42.393944 15325 raft_consensus.cc:2229] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:42.394187 15326 raft_consensus.cc:2229] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:42.394203 14710 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 04:40:42.394491 15326 raft_consensus.cc:2258] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 04:40:42.394580 14710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:42.394835 15328 tablet_replica.cc:330] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7: stopping tablet replica
I20260809 04:40:42.395100 15328 raft_consensus.cc:2229] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:42.395462 15328 raft_consensus.cc:2258] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:42.395843 15325 raft_consensus.cc:2258] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:42.396880 15326 ts_tablet_manager.cc:1905] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:42.399241 14871 tablet_service.cc:1467] Processing CreateTablet for tablet 4891acba3ab040a680c8bd8dd2e974ee (DEFAULT_TABLE table=test-table [id=d2a813fd828e4ac5aa776862dc00338e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:42.399515 14871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4891acba3ab040a680c8bd8dd2e974ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:42.399966 15326 ts_tablet_manager.cc:1918] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:42.400059 15326 log.cc:1196] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/wal/wals/2323a4be46aa42b59725f42da73ca48b
I20260809 04:40:42.400367 15326 ts_tablet_manager.cc:1939] T 2323a4be46aa42b59725f42da73ca48b P 173a296a031a4a8cb49dfd26f3bd0ca7: Deleting consensus metadata
I20260809 04:40:42.401383 15032 tablet_service.cc:1467] Processing CreateTablet for tablet 4891acba3ab040a680c8bd8dd2e974ee (DEFAULT_TABLE table=test-table [id=d2a813fd828e4ac5aa776862dc00338e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:42.401643 15032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4891acba3ab040a680c8bd8dd2e974ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:42.403482 15300 tablet_bootstrap.cc:492] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614: Bootstrap starting.
I20260809 04:40:42.404156 14694 catalog_manager.cc:4841] TS 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341): tablet 2323a4be46aa42b59725f42da73ca48b (table test-table [id=3041661474a848e281622f7f593803b3]) successfully deleted
I20260809 04:40:42.404475 15300 tablet_bootstrap.cc:654] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:42.404701 15184 tablet_service.cc:1467] Processing CreateTablet for tablet 4891acba3ab040a680c8bd8dd2e974ee (DEFAULT_TABLE table=test-table [id=d2a813fd828e4ac5aa776862dc00338e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:42.404946 15184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4891acba3ab040a680c8bd8dd2e974ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:42.405691 15300 tablet_bootstrap.cc:492] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614: No bootstrap required, opened a new log
I20260809 04:40:42.405714 15299 tablet_bootstrap.cc:492] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7: Bootstrap starting.
I20260809 04:40:42.405771 15300 ts_tablet_manager.cc:1397] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:42.406522 15300 raft_consensus.cc:348] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.406648 15300 raft_consensus.cc:374] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:42.406690 15300 raft_consensus.cc:729] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be696c8e15ee488e9915ed6330f3f614, State: Initialized, Role: FOLLOWER
I20260809 04:40:42.406711 15299 tablet_bootstrap.cc:654] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:42.406828 15300 consensus_queue.cc:260] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.407111 15300 ts_tablet_manager.cc:1428] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:42.407639 15298 tablet_bootstrap.cc:492] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7: Bootstrap starting.
I20260809 04:40:42.408524 15298 tablet_bootstrap.cc:654] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:42.408788 15328 ts_tablet_manager.cc:1905] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:42.409524 15325 ts_tablet_manager.cc:1905] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:42.410249 15298 tablet_bootstrap.cc:492] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7: No bootstrap required, opened a new log
I20260809 04:40:42.410338 15298 ts_tablet_manager.cc:1397] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 04:40:42.410751 15298 raft_consensus.cc:348] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.410861 15298 raft_consensus.cc:374] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:42.410893 15298 raft_consensus.cc:729] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66897116c1df45759027f5fbf9c200f7, State: Initialized, Role: FOLLOWER
I20260809 04:40:42.411026 15298 consensus_queue.cc:260] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.411262 15298 ts_tablet_manager.cc:1428] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:42.412313 15325 ts_tablet_manager.cc:1918] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:42.412425 15325 log.cc:1196] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/2323a4be46aa42b59725f42da73ca48b
I20260809 04:40:42.412743 15325 ts_tablet_manager.cc:1939] T 2323a4be46aa42b59725f42da73ca48b P be696c8e15ee488e9915ed6330f3f614: Deleting consensus metadata
I20260809 04:40:42.413981 14696 catalog_manager.cc:4841] TS be696c8e15ee488e9915ed6330f3f614 (127.5.222.194:32899): tablet 2323a4be46aa42b59725f42da73ca48b (table test-table [id=3041661474a848e281622f7f593803b3]) successfully deleted
I20260809 04:40:42.415302 15328 ts_tablet_manager.cc:1918] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:42.415397 15328 log.cc:1196] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/2323a4be46aa42b59725f42da73ca48b
I20260809 04:40:42.415694 15328 ts_tablet_manager.cc:1939] T 2323a4be46aa42b59725f42da73ca48b P 66897116c1df45759027f5fbf9c200f7: Deleting consensus metadata
I20260809 04:40:42.416697 14693 catalog_manager.cc:4841] TS 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803): tablet 2323a4be46aa42b59725f42da73ca48b (table test-table [id=3041661474a848e281622f7f593803b3]) successfully deleted
I20260809 04:40:42.417356 15299 tablet_bootstrap.cc:492] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7: No bootstrap required, opened a new log
I20260809 04:40:42.417476 15299 ts_tablet_manager.cc:1397] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260809 04:40:42.418115 15299 raft_consensus.cc:348] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.418237 15299 raft_consensus.cc:374] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:42.418296 15299 raft_consensus.cc:729] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173a296a031a4a8cb49dfd26f3bd0ca7, State: Initialized, Role: FOLLOWER
I20260809 04:40:42.418480 15299 consensus_queue.cc:260] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.418792 15299 ts_tablet_manager.cc:1428] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:42.493533 15307 raft_consensus.cc:482] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:42.493706 15307 raft_consensus.cc:504] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.494973 15307 leader_election.cc:288] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341), 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803)
I20260809 04:40:42.500043 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.495429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:44451 (local address 127.5.222.195:42803)
0809 04:40:42.495565 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.495572 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.495645 (+ 73us) server_negotiation.cc:407] Connection header received
0809 04:40:42.495884 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.495890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.495943 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.495995 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:42.496531 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.497417 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.498973 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.499385 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.499530 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.499591 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.499825 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.499917 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:40:42.500108 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.495213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:42341 (local address 127.5.222.194:52229)
0809 04:40:42.495823 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.495837 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.495928 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.496188 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.496190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.496201 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:42.496323 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.496330 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.497364 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.497370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.498807 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.498820 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.499690 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.499694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.499805 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.499825 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.499895 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.499962 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":102,"thread_start_us":113,"threads_started":1}
I20260809 04:40:42.500144 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.495694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:52229 (local address 127.5.222.193:42341)
0809 04:40:42.495836 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.495840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.495927 (+ 87us) server_negotiation.cc:407] Connection header received
0809 04:40:42.496020 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.496023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.496074 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.496163 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:42.496534 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.497251 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.498965 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.499578 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.499715 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.499767 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.499937 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.500018 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:40:42.500542 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.495213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:42803 (local address 127.5.222.194:44451)
0809 04:40:42.495547 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.495563 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.495740 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.496165 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.496169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.496188 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:42.496327 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.496331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.497549 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.497553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.498815 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.498821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.499536 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.499540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.499657 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.499686 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.500254 (+ 568us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.500354 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":23,"spinlock_wait_cycles":5376,"thread_start_us":76,"threads_started":1}
I20260809 04:40:42.500591 14895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4891acba3ab040a680c8bd8dd2e974ee" candidate_uuid: "be696c8e15ee488e9915ed6330f3f614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" is_pre_election: true
I20260809 04:40:42.500743 14895 raft_consensus.cc:2454] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be696c8e15ee488e9915ed6330f3f614 in term 0.
I20260809 04:40:42.501194 14976 leader_election.cc:302] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173a296a031a4a8cb49dfd26f3bd0ca7, be696c8e15ee488e9915ed6330f3f614; no voters:
I20260809 04:40:42.501245 15211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4891acba3ab040a680c8bd8dd2e974ee" candidate_uuid: "be696c8e15ee488e9915ed6330f3f614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66897116c1df45759027f5fbf9c200f7" is_pre_election: true
I20260809 04:40:42.501480 15307 raft_consensus.cc:2781] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:42.501562 15307 raft_consensus.cc:482] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:42.501497 15211 raft_consensus.cc:2454] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be696c8e15ee488e9915ed6330f3f614 in term 0.
I20260809 04:40:42.501613 15307 raft_consensus.cc:3037] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:42.502959 15307 raft_consensus.cc:504] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.503350 15307 leader_election.cc:288] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [CANDIDATE]: Term 1 election: Requested vote from peers 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341), 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803)
I20260809 04:40:42.503659 15211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4891acba3ab040a680c8bd8dd2e974ee" candidate_uuid: "be696c8e15ee488e9915ed6330f3f614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66897116c1df45759027f5fbf9c200f7"
I20260809 04:40:42.503659 14895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4891acba3ab040a680c8bd8dd2e974ee" candidate_uuid: "be696c8e15ee488e9915ed6330f3f614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7"
I20260809 04:40:42.503775 14895 raft_consensus.cc:3037] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:42.503783 15211 raft_consensus.cc:3037] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:42.504962 14895 raft_consensus.cc:2454] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be696c8e15ee488e9915ed6330f3f614 in term 1.
I20260809 04:40:42.504962 15211 raft_consensus.cc:2454] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be696c8e15ee488e9915ed6330f3f614 in term 1.
I20260809 04:40:42.505308 14975 leader_election.cc:302] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66897116c1df45759027f5fbf9c200f7, be696c8e15ee488e9915ed6330f3f614; no voters:
I20260809 04:40:42.505527 15307 raft_consensus.cc:2781] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:42.505780 15307 raft_consensus.cc:686] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 1 LEADER]: Becoming Leader. State: Replica: be696c8e15ee488e9915ed6330f3f614, State: Running, Role: LEADER
I20260809 04:40:42.506078 15307 consensus_queue.cc:237] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.508818 14710 catalog_manager.cc:5495] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 reported cstate change: term changed from 0 to 1, leader changed from <none> to be696c8e15ee488e9915ed6330f3f614 (127.5.222.194). New cstate: current_term: 1 leader_uuid: "be696c8e15ee488e9915ed6330f3f614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } health_report { overall_health: UNKNOWN } } }
Aug 09 04:40:42 dist-test-slave-vxj2 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250441, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.254@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:42.555667 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.546983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40455 (local address 127.0.0.1:57062)
0809 04:40:42.547339 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.547352 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.547484 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.547914 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.547918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.548316 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:42.548418 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.548425 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.549245 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.549253 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.549815 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.549823 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.550306 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.550308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.550374 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.550906 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:42.550920 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:42.552540 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:42.554135 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:42.554144 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:42.554149 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:42.554418 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:42.554672 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:42.554674 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:42.554675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:42.554757 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:42.555106 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:42.555116 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:42.555252 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.555367 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.555527 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":126,"threads_started":1}
I20260809 04:40:42.555738 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.547107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57062 (local address 127.5.222.254:40455)
0809 04:40:42.547260 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.547265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.547462 (+ 197us) server_negotiation.cc:407] Connection header received
0809 04:40:42.547562 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.547565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.547618 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.547717 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:42.548541 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.549073 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.549967 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.550180 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.550270 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.552676 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:42.552679 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:42.552680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:42.552701 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:42.554030 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:42.554505 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:42.554508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:42.554509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:42.554579 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:42.554831 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:42.554833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:42.554834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:42.554998 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:42.555071 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.555443 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.555601 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":55}
I20260809 04:40:42.559516 14710 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:57062:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 04:40:42.560254 14710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:42.565107 14871 tablet_service.cc:1467] Processing CreateTablet for tablet baf96664ed674f8db0405c5c10ef745d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2174a9faa4c1433c8a345f7f4c793c07]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:40:42.565201 15032 tablet_service.cc:1467] Processing CreateTablet for tablet baf96664ed674f8db0405c5c10ef745d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2174a9faa4c1433c8a345f7f4c793c07]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:40:42.565304 15184 tablet_service.cc:1467] Processing CreateTablet for tablet baf96664ed674f8db0405c5c10ef745d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2174a9faa4c1433c8a345f7f4c793c07]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:40:42.565613 14871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baf96664ed674f8db0405c5c10ef745d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:42.565608 15032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baf96664ed674f8db0405c5c10ef745d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:42.565616 15184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baf96664ed674f8db0405c5c10ef745d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:42.568460 15299 tablet_bootstrap.cc:492] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: Bootstrap starting.
I20260809 04:40:42.568478 15300 tablet_bootstrap.cc:492] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614: Bootstrap starting.
I20260809 04:40:42.568635 15298 tablet_bootstrap.cc:492] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7: Bootstrap starting.
I20260809 04:40:42.569510 15299 tablet_bootstrap.cc:654] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:42.569511 15298 tablet_bootstrap.cc:654] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:42.569530 15300 tablet_bootstrap.cc:654] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:42.570817 15299 tablet_bootstrap.cc:492] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: No bootstrap required, opened a new log
I20260809 04:40:42.570919 15299 ts_tablet_manager.cc:1397] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:42.571358 15299 raft_consensus.cc:348] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.571513 15299 raft_consensus.cc:374] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:42.571552 15299 raft_consensus.cc:729] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173a296a031a4a8cb49dfd26f3bd0ca7, State: Initialized, Role: FOLLOWER
I20260809 04:40:42.571671 15299 consensus_queue.cc:260] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.571817 15298 tablet_bootstrap.cc:492] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7: No bootstrap required, opened a new log
I20260809 04:40:42.571892 15298 ts_tablet_manager.cc:1397] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:42.571973 15306 tablet_replica.cc:439] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } }
I20260809 04:40:42.572048 15300 tablet_bootstrap.cc:492] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614: No bootstrap required, opened a new log
I20260809 04:40:42.572085 15306 tablet_replica.cc:442] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:42.572126 15300 ts_tablet_manager.cc:1397] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 04:40:42.572292 15298 raft_consensus.cc:348] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.572397 15298 raft_consensus.cc:374] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:42.572444 15298 raft_consensus.cc:729] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66897116c1df45759027f5fbf9c200f7, State: Initialized, Role: FOLLOWER
I20260809 04:40:42.572564 15298 consensus_queue.cc:260] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.572548 15300 raft_consensus.cc:348] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.572650 15300 raft_consensus.cc:374] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:42.572682 15300 raft_consensus.cc:729] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be696c8e15ee488e9915ed6330f3f614, State: Initialized, Role: FOLLOWER
I20260809 04:40:42.572795 15300 consensus_queue.cc:260] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.572867 15298 ts_tablet_manager.cc:1428] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:42.573050 15300 ts_tablet_manager.cc:1428] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:40:42.573136 15307 tablet_replica.cc:439] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } }
I20260809 04:40:42.573287 15307 tablet_replica.cc:442] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:42.573518 15322 tablet_replica.cc:439] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } }
I20260809 04:40:42.573670 15322 tablet_replica.cc:442] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:42.574499 15299 ts_tablet_manager.cc:1428] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260809 04:40:42.690515 15306 raft_consensus.cc:482] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:42.690722 15306 raft_consensus.cc:504] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.691972 15306 leader_election.cc:288] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be696c8e15ee488e9915ed6330f3f614 (127.5.222.194:32899), 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803)
I20260809 04:40:42.696307 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.692332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:34697 (local address 127.5.222.195:42803)
0809 04:40:42.692477 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.692483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.692794 (+ 311us) server_negotiation.cc:407] Connection header received
0809 04:40:42.692964 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.692966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.693005 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.693103 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:42.693520 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.694001 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.695328 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.695726 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.695851 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.695904 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.696083 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.696175 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:40:42.696331 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.692210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:32899 (local address 127.5.222.193:52835)
0809 04:40:42.692612 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.692623 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.692755 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.693101 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.693104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.693121 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:42.693318 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.693323 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.694102 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.694106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.695114 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.695124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.695870 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.695874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.696011 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.696051 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.696130 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.696191 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":161,"spinlock_wait_cycles":64128,"thread_start_us":138,"threads_started":1}
I20260809 04:40:42.696355 15349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.692210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:42803 (local address 127.5.222.193:34697)
0809 04:40:42.692711 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.692723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.692849 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.693140 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.693143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.693232 (+ 89us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:40:42.693325 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.693330 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.694141 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.694143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.695110 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.695119 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.695847 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.695851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.695967 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.695985 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.696045 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.696093 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":134,"spinlock_wait_cycles":768,"thread_start_us":117,"threads_started":1}
I20260809 04:40:42.696403 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.692323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:52835 (local address 127.5.222.194:32899)
0809 04:40:42.692468 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.692473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.692717 (+ 244us) server_negotiation.cc:407] Connection header received
0809 04:40:42.692898 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.692903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.692969 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.693052 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:40:42.693442 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.693987 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.695354 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.695733 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.695862 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.695912 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.696213 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.696285 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:40:42.696801 15052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baf96664ed674f8db0405c5c10ef745d" candidate_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be696c8e15ee488e9915ed6330f3f614" is_pre_election: true
I20260809 04:40:42.696954 15052 raft_consensus.cc:2454] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 173a296a031a4a8cb49dfd26f3bd0ca7 in term 0.
I20260809 04:40:42.697033 15211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baf96664ed674f8db0405c5c10ef745d" candidate_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66897116c1df45759027f5fbf9c200f7" is_pre_election: true
I20260809 04:40:42.697191 15211 raft_consensus.cc:2454] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 173a296a031a4a8cb49dfd26f3bd0ca7 in term 0.
I20260809 04:40:42.697337 14816 leader_election.cc:302] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173a296a031a4a8cb49dfd26f3bd0ca7, be696c8e15ee488e9915ed6330f3f614; no voters:
I20260809 04:40:42.697604 15306 raft_consensus.cc:2781] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:42.697660 15306 raft_consensus.cc:482] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:42.697697 15306 raft_consensus.cc:3037] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:42.698773 15306 raft_consensus.cc:504] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.699146 15306 leader_election.cc:288] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [CANDIDATE]: Term 1 election: Requested vote from peers be696c8e15ee488e9915ed6330f3f614 (127.5.222.194:32899), 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803)
I20260809 04:40:42.699507 15211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baf96664ed674f8db0405c5c10ef745d" candidate_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66897116c1df45759027f5fbf9c200f7"
I20260809 04:40:42.699656 15211 raft_consensus.cc:3037] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:42.700634 15211 raft_consensus.cc:2454] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 173a296a031a4a8cb49dfd26f3bd0ca7 in term 1.
I20260809 04:40:42.700934 15052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baf96664ed674f8db0405c5c10ef745d" candidate_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be696c8e15ee488e9915ed6330f3f614"
I20260809 04:40:42.700973 14812 leader_election.cc:302] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173a296a031a4a8cb49dfd26f3bd0ca7, 66897116c1df45759027f5fbf9c200f7; no voters:
I20260809 04:40:42.701071 15052 raft_consensus.cc:3037] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:42.701190 15306 raft_consensus.cc:2781] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:42.701447 15306 raft_consensus.cc:686] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 LEADER]: Becoming Leader. State: Replica: 173a296a031a4a8cb49dfd26f3bd0ca7, State: Running, Role: LEADER
I20260809 04:40:42.701706 15306 consensus_queue.cc:237] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.702239 15052 raft_consensus.cc:2454] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 173a296a031a4a8cb49dfd26f3bd0ca7 in term 1.
I20260809 04:40:42.703197 15350 tablet_replica.cc:439] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: TxnStatusTablet state changed. Reason: New leader 173a296a031a4a8cb49dfd26f3bd0ca7. Latest consensus state: current_term: 1 leader_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } }
I20260809 04:40:42.703315 15350 tablet_replica.cc:442] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:40:42.703820 15352 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 04:40:42.703991 15352 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 04:40:42.704540 14711 catalog_manager.cc:5495] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193). New cstate: current_term: 1 leader_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } health_report { overall_health: UNKNOWN } } }
I20260809 04:40:42.731086 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.726969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33178 (local address 127.5.222.193:42341)
0809 04:40:42.727107 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.727111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.727164 (+ 53us) server_negotiation.cc:407] Connection header received
0809 04:40:42.727371 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.727373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.727429 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.727512 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:42.727793 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.728285 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.729574 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.729786 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.729899 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.730229 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:42.730659 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:42.730789 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.730870 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.730939 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 04:40:42.731086 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.726864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:42341 (local address 127.0.0.1:33178)
0809 04:40:42.727107 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.727122 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.727227 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.727551 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.727554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.727562 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:40:42.727681 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.727687 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.728392 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.728396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.729448 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.729459 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.729904 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.729907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.730004 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.730058 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:40:42.730766 (+ 708us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:40:42.730787 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.730859 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.730952 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260809 04:40:42.734290 15052 raft_consensus.cc:1261] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Refusing update from remote peer 173a296a031a4a8cb49dfd26f3bd0ca7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:42.734477 15211 raft_consensus.cc:1261] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Refusing update from remote peer 173a296a031a4a8cb49dfd26f3bd0ca7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:42.734818 15350 consensus_queue.cc:1035] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:42.735019 15306 consensus_queue.cc:1035] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:42.736748 15307 tablet_replica.cc:439] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614: TxnStatusTablet state changed. Reason: New leader 173a296a031a4a8cb49dfd26f3bd0ca7. Latest consensus state: current_term: 1 leader_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } }
I20260809 04:40:42.736864 15307 tablet_replica.cc:442] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:42.736801 15322 tablet_replica.cc:439] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7: TxnStatusTablet state changed. Reason: New leader 173a296a031a4a8cb49dfd26f3bd0ca7. Latest consensus state: current_term: 1 leader_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } }
I20260809 04:40:42.736904 15322 tablet_replica.cc:442] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:42.739418 15306 tablet_replica.cc:439] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } }
I20260809 04:40:42.739521 15306 tablet_replica.cc:442] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:40:42.739645 15314 mvcc.cc:205] Tried to move back new op lower bound from 7316481813434441728 to 7316481813308989440. Current Snapshot: MvccSnapshot[applied={T|T < 7316481813434441728 or (T in {7316481813434441728})}]
I20260809 04:40:42.741480 15307 tablet_replica.cc:439] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } }
I20260809 04:40:42.741578 15307 tablet_replica.cc:442] T baf96664ed674f8db0405c5c10ef745d P be696c8e15ee488e9915ed6330f3f614: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:42.743150 15322 tablet_replica.cc:439] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } }
I20260809 04:40:42.743278 15322 tablet_replica.cc:442] T baf96664ed674f8db0405c5c10ef745d P 66897116c1df45759027f5fbf9c200f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:40:42.743782 15350 tablet_replica.cc:439] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } } }
I20260809 04:40:42.743894 15350 tablet_replica.cc:442] T baf96664ed674f8db0405c5c10ef745d P 173a296a031a4a8cb49dfd26f3bd0ca7: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:40:42.743167 15313 mvcc.cc:205] Tried to move back new op lower bound from 7316481813434441728 to 7316481813308989440. Current Snapshot: MvccSnapshot[applied={T|T < 7316481813434441728}]
I20260809 04:40:42.748193 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.744175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56418 (local address 127.5.222.194:32899)
0809 04:40:42.744285 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.744289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.744302 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:40:42.744367 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.744370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.744427 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.744521 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:42.744868 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.745673 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.746869 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.747084 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.747165 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.747433 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:42.747804 (+ 371us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:42.747887 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.748004 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.748088 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 04:40:42.760394 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.754788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:41377 (local address 127.5.222.193:42341)
0809 04:40:42.754924 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.754930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.757445 (+ 2515us) server_negotiation.cc:407] Connection header received
0809 04:40:42.757592 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.757595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.757645 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.757723 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:42.757975 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.758464 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.759331 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.759550 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.759659 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.759942 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:42.760007 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:42.760088 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.760212 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.760259 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:40:42.760682 15359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.754697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:42341 (local address 127.5.222.194:41377)
0809 04:40:42.757357 (+ 2660us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.757369 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.757495 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.757743 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.757747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.757754 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:40:42.757865 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.757869 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.758577 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.758582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.759207 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.759215 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.759661 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.759664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.759757 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.759796 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:40:42.760091 (+ 295us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:40:42.760100 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.760188 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.760264 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2574,"thread_start_us":115,"threads_started":1}
I20260809 04:40:42.767280 15211 raft_consensus.cc:1261] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Refusing update from remote peer be696c8e15ee488e9915ed6330f3f614: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:42.767445 14895 raft_consensus.cc:1261] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Refusing update from remote peer be696c8e15ee488e9915ed6330f3f614: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:42.767776 15307 consensus_queue.cc:1035] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:42.768046 15335 consensus_queue.cc:1035] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:42.791553 15365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.787531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:32899 (local address 127.5.222.193:39581)
0809 04:40:42.787825 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:42.787847 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:42.787934 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:42.788142 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:42.788144 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:42.788149 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:40:42.788246 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.788250 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.789442 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.789447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.790296 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:42.790310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.790902 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.790905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:42.790983 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.791024 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:40:42.791309 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:40:42.791313 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:40:42.791361 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 04:40:42.791426 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":107,"threads_started":1}
I20260809 04:40:42.791570 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.787629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:39581 (local address 127.5.222.194:32899)
0809 04:40:42.787733 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.787746 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.787917 (+ 171us) server_negotiation.cc:407] Connection header received
0809 04:40:42.788005 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.788008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.788054 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.788141 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:42.788332 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.789128 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.790430 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.790781 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.790887 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.791148 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:42.791226 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:42.791300 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.791373 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.791424 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 04:40:42.829989 14710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:42.830174 14710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:42.832556 14710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655: Sending DeleteTablet for 3 replicas of tablet 4891acba3ab040a680c8bd8dd2e974ee
I20260809 04:40:42.833082 14871 tablet_service.cc:1514] Processing DeleteTablet for tablet 4891acba3ab040a680c8bd8dd2e974ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:42 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:33172
I20260809 04:40:42.833308 15326 tablet_replica.cc:330] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7: stopping tablet replica
I20260809 04:40:42.833282 15032 tablet_service.cc:1514] Processing DeleteTablet for tablet 4891acba3ab040a680c8bd8dd2e974ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:42 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:56406
I20260809 04:40:42.833451 15326 raft_consensus.cc:2229] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:42.833585 15326 raft_consensus.cc:2258] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:42.833603 15325 tablet_replica.cc:330] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614: stopping tablet replica
I20260809 04:40:42.833714 15325 raft_consensus.cc:2229] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:42.834028 15325 raft_consensus.cc:2258] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:42.834501 15184 tablet_service.cc:1514] Processing DeleteTablet for tablet 4891acba3ab040a680c8bd8dd2e974ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:42 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:53336
I20260809 04:40:42.834560 15326 ts_tablet_manager.cc:1905] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:42.834678 15328 tablet_replica.cc:330] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7: stopping tablet replica
I20260809 04:40:42.834810 15328 raft_consensus.cc:2229] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:42.834924 15328 raft_consensus.cc:2258] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:42.837607 15326 ts_tablet_manager.cc:1918] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:42.837687 15326 log.cc:1196] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/wal/wals/4891acba3ab040a680c8bd8dd2e974ee
I20260809 04:40:42.837951 15326 ts_tablet_manager.cc:1939] T 4891acba3ab040a680c8bd8dd2e974ee P 173a296a031a4a8cb49dfd26f3bd0ca7: Deleting consensus metadata
I20260809 04:40:42.838670 15328 ts_tablet_manager.cc:1905] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:42.838845 14694 catalog_manager.cc:4841] TS 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341): tablet 4891acba3ab040a680c8bd8dd2e974ee (table test-table [id=d2a813fd828e4ac5aa776862dc00338e]) successfully deleted
I20260809 04:40:42.841327 15325 ts_tablet_manager.cc:1905] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:42.841856 15328 ts_tablet_manager.cc:1918] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:42.841930 15328 log.cc:1196] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/4891acba3ab040a680c8bd8dd2e974ee
I20260809 04:40:42.842206 15328 ts_tablet_manager.cc:1939] T 4891acba3ab040a680c8bd8dd2e974ee P 66897116c1df45759027f5fbf9c200f7: Deleting consensus metadata
I20260809 04:40:42.843099 14693 catalog_manager.cc:4841] TS 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803): tablet 4891acba3ab040a680c8bd8dd2e974ee (table test-table [id=d2a813fd828e4ac5aa776862dc00338e]) successfully deleted
I20260809 04:40:42.843993 15325 ts_tablet_manager.cc:1918] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:42.844074 15325 log.cc:1196] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/4891acba3ab040a680c8bd8dd2e974ee
I20260809 04:40:42.844328 15325 ts_tablet_manager.cc:1939] T 4891acba3ab040a680c8bd8dd2e974ee P be696c8e15ee488e9915ed6330f3f614: Deleting consensus metadata
I20260809 04:40:42.846833 14696 catalog_manager.cc:4841] TS be696c8e15ee488e9915ed6330f3f614 (127.5.222.194:32899): tablet 4891acba3ab040a680c8bd8dd2e974ee (table test-table [id=d2a813fd828e4ac5aa776862dc00338e]) successfully deleted
Aug 09 04:40:42 dist-test-slave-vxj2 krb5kdc[14663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:40:42 dist-test-slave-vxj2 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:42 dist-test-slave-vxj2 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:40:42.878069 14767 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:42.869493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57076 (local address 127.5.222.254:40455)
0809 04:40:42.869624 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.869629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.869997 (+ 368us) server_negotiation.cc:407] Connection header received
0809 04:40:42.870137 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.870140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.870197 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.870291 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:42.871245 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.872087 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.873382 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.873740 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.873849 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.877902 (+ 4053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57076: BlockingRecv error: recv got EOF from 127.0.0.1:57076 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 04:40:42 dist-test-slave-vxj2 krb5kdc[14663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM
I20260809 04:40:42.894825 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:42.881574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57084 (local address 127.5.222.254:40455)
0809 04:40:42.881684 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:42.881688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:42.886097 (+ 4409us) server_negotiation.cc:407] Connection header received
0809 04:40:42.886189 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:42.886192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:42.886244 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:42.886334 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:42.887073 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.887885 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.888797 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:42.889134 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:42.889263 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:42.891634 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:42.891638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:42.891641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:42.891669 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:42.893245 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:42.893640 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:42.893643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:42.893645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:42.893735 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:42.893985 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:42.893988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:42.893990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:42.894211 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:42.894348 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:42.894539 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 04:40:42.894656 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 04:40:42.897821 14710 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:42.898334 14710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:42.903155 14871 tablet_service.cc:1467] Processing CreateTablet for tablet ca81ae7de1fd444dbab0573bc43a5a84 (DEFAULT_TABLE table=test-table [id=1b5871ac21b942739809b1e1af6cad86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:42.903520 14871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca81ae7de1fd444dbab0573bc43a5a84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:42.903545 15032 tablet_service.cc:1467] Processing CreateTablet for tablet ca81ae7de1fd444dbab0573bc43a5a84 (DEFAULT_TABLE table=test-table [id=1b5871ac21b942739809b1e1af6cad86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:42.903822 15032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca81ae7de1fd444dbab0573bc43a5a84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:42.905663 15299 tablet_bootstrap.cc:492] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7: Bootstrap starting.
I20260809 04:40:42.906065 15184 tablet_service.cc:1467] Processing CreateTablet for tablet ca81ae7de1fd444dbab0573bc43a5a84 (DEFAULT_TABLE table=test-table [id=1b5871ac21b942739809b1e1af6cad86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:42.906333 15184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca81ae7de1fd444dbab0573bc43a5a84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:42.906347 15300 tablet_bootstrap.cc:492] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614: Bootstrap starting.
I20260809 04:40:42.906543 15299 tablet_bootstrap.cc:654] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:42.907254 15300 tablet_bootstrap.cc:654] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:42.908489 15300 tablet_bootstrap.cc:492] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614: No bootstrap required, opened a new log
I20260809 04:40:42.908574 15300 ts_tablet_manager.cc:1397] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 04:40:42.908610 15298 tablet_bootstrap.cc:492] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7: Bootstrap starting.
I20260809 04:40:42.909132 15300 raft_consensus.cc:348] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.909260 15300 raft_consensus.cc:374] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:42.909291 15300 raft_consensus.cc:729] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be696c8e15ee488e9915ed6330f3f614, State: Initialized, Role: FOLLOWER
I20260809 04:40:42.909422 15300 consensus_queue.cc:260] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.909539 15298 tablet_bootstrap.cc:654] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:42.909683 15300 ts_tablet_manager.cc:1428] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 04:40:42.910627 15299 tablet_bootstrap.cc:492] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7: No bootstrap required, opened a new log
I20260809 04:40:42.910742 15299 ts_tablet_manager.cc:1397] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 04:40:42.911191 15299 raft_consensus.cc:348] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.911325 15299 raft_consensus.cc:374] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:42.911361 15299 raft_consensus.cc:729] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173a296a031a4a8cb49dfd26f3bd0ca7, State: Initialized, Role: FOLLOWER
I20260809 04:40:42.911504 15299 consensus_queue.cc:260] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.911844 15299 ts_tablet_manager.cc:1428] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 04:40:42.912575 15298 tablet_bootstrap.cc:492] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7: No bootstrap required, opened a new log
I20260809 04:40:42.912671 15298 ts_tablet_manager.cc:1397] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 04:40:42.913064 15298 raft_consensus.cc:348] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.913203 15298 raft_consensus.cc:374] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:42.913265 15298 raft_consensus.cc:729] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66897116c1df45759027f5fbf9c200f7, State: Initialized, Role: FOLLOWER
I20260809 04:40:42.913407 15298 consensus_queue.cc:260] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.913691 15298 ts_tablet_manager.cc:1428] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 04:40:42.983115 15363 raft_consensus.cc:482] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:42.983312 15363 raft_consensus.cc:504] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.983845 15363 leader_election.cc:288] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341), 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803)
I20260809 04:40:42.984292 15211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca81ae7de1fd444dbab0573bc43a5a84" candidate_uuid: "be696c8e15ee488e9915ed6330f3f614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66897116c1df45759027f5fbf9c200f7" is_pre_election: true
I20260809 04:40:42.984315 14895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca81ae7de1fd444dbab0573bc43a5a84" candidate_uuid: "be696c8e15ee488e9915ed6330f3f614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" is_pre_election: true
I20260809 04:40:42.984468 15211 raft_consensus.cc:2454] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be696c8e15ee488e9915ed6330f3f614 in term 0.
I20260809 04:40:42.984491 14895 raft_consensus.cc:2454] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be696c8e15ee488e9915ed6330f3f614 in term 0.
I20260809 04:40:42.984813 14976 leader_election.cc:302] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173a296a031a4a8cb49dfd26f3bd0ca7, be696c8e15ee488e9915ed6330f3f614; no voters:
I20260809 04:40:42.985090 15363 raft_consensus.cc:2781] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:42.985183 15363 raft_consensus.cc:482] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:42.985250 15363 raft_consensus.cc:3037] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:42.986583 15363 raft_consensus.cc:504] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.986996 15363 leader_election.cc:288] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [CANDIDATE]: Term 1 election: Requested vote from peers 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341), 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803)
I20260809 04:40:42.987401 14895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca81ae7de1fd444dbab0573bc43a5a84" candidate_uuid: "be696c8e15ee488e9915ed6330f3f614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7"
I20260809 04:40:42.987455 15211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca81ae7de1fd444dbab0573bc43a5a84" candidate_uuid: "be696c8e15ee488e9915ed6330f3f614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66897116c1df45759027f5fbf9c200f7"
I20260809 04:40:42.987550 14895 raft_consensus.cc:3037] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:42.987568 15211 raft_consensus.cc:3037] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:42.988878 14895 raft_consensus.cc:2454] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be696c8e15ee488e9915ed6330f3f614 in term 1.
I20260809 04:40:42.988878 15211 raft_consensus.cc:2454] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be696c8e15ee488e9915ed6330f3f614 in term 1.
I20260809 04:40:42.989264 14976 leader_election.cc:302] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173a296a031a4a8cb49dfd26f3bd0ca7, be696c8e15ee488e9915ed6330f3f614; no voters:
I20260809 04:40:42.989450 15363 raft_consensus.cc:2781] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:42.989599 15363 raft_consensus.cc:686] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 1 LEADER]: Becoming Leader. State: Replica: be696c8e15ee488e9915ed6330f3f614, State: Running, Role: LEADER
I20260809 04:40:42.989741 15363 consensus_queue.cc:237] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:42.991603 14709 catalog_manager.cc:5495] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 reported cstate change: term changed from 0 to 1, leader changed from <none> to be696c8e15ee488e9915ed6330f3f614 (127.5.222.194). New cstate: current_term: 1 leader_uuid: "be696c8e15ee488e9915ed6330f3f614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } health_report { overall_health: UNKNOWN } } }
I20260809 04:40:43.020113 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:43.016757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56422 (local address 127.5.222.194:32899)
0809 04:40:43.016927 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:43.016932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:43.016953 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:40:43.016997 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:43.016999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:43.017041 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:43.017126 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:40:43.017435 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:43.017957 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:43.018964 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:43.019226 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:43.019358 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:43.019666 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:40:43.019738 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:40:43.019820 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:43.019918 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 04:40:43.019971 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:40:43.022253 15211 raft_consensus.cc:1261] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Refusing update from remote peer be696c8e15ee488e9915ed6330f3f614: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:43.022297 14895 raft_consensus.cc:1261] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Refusing update from remote peer be696c8e15ee488e9915ed6330f3f614: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:43.022742 15363 consensus_queue.cc:1035] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:43.022941 15335 consensus_queue.cc:1035] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:43.030424 14710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:43.030592 14710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:43.033387 14710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655: Sending DeleteTablet for 3 replicas of tablet ca81ae7de1fd444dbab0573bc43a5a84
I20260809 04:40:43.033919 14871 tablet_service.cc:1514] Processing DeleteTablet for tablet ca81ae7de1fd444dbab0573bc43a5a84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:43 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:33172
I20260809 04:40:43.034111 15326 tablet_replica.cc:330] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7: stopping tablet replica
I20260809 04:40:43.034108 15184 tablet_service.cc:1514] Processing DeleteTablet for tablet ca81ae7de1fd444dbab0573bc43a5a84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:43 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:53336
I20260809 04:40:43.034212 15032 tablet_service.cc:1514] Processing DeleteTablet for tablet ca81ae7de1fd444dbab0573bc43a5a84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:43 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:56406
I20260809 04:40:43.034252 15326 raft_consensus.cc:2229] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:43.034307 15031 tablet_service.cc:1514] Processing DeleteTablet for tablet ca81ae7de1fd444dbab0573bc43a5a84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:43 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:56406
I20260809 04:40:43.034389 15326 raft_consensus.cc:2258] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:43.034514 15325 tablet_replica.cc:330] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614: stopping tablet replica
I20260809 04:40:43.034638 15325 raft_consensus.cc:2229] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:43.034833 15325 raft_consensus.cc:2258] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:43.035055 15328 tablet_replica.cc:330] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7: stopping tablet replica
I20260809 04:40:43.035133 15326 ts_tablet_manager.cc:1905] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:43.035205 15328 raft_consensus.cc:2229] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:43.035346 15328 raft_consensus.cc:2258] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:43.035542 15325 ts_tablet_manager.cc:1905] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:43.036176 15328 ts_tablet_manager.cc:1905] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:43.037415 15326 ts_tablet_manager.cc:1918] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:43.037528 15326 log.cc:1196] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-0/wal/wals/ca81ae7de1fd444dbab0573bc43a5a84
I20260809 04:40:43.037796 15326 ts_tablet_manager.cc:1939] T ca81ae7de1fd444dbab0573bc43a5a84 P 173a296a031a4a8cb49dfd26f3bd0ca7: Deleting consensus metadata
I20260809 04:40:43.038167 15325 ts_tablet_manager.cc:1918] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:43.038270 15325 log.cc:1196] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/ca81ae7de1fd444dbab0573bc43a5a84
I20260809 04:40:43.038515 14694 catalog_manager.cc:4841] TS 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341): tablet ca81ae7de1fd444dbab0573bc43a5a84 (table test-table [id=1b5871ac21b942739809b1e1af6cad86]) successfully deleted
I20260809 04:40:43.038846 15328 ts_tablet_manager.cc:1918] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:40:43.038945 15328 log.cc:1196] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/ca81ae7de1fd444dbab0573bc43a5a84
I20260809 04:40:43.039160 15325 ts_tablet_manager.cc:1939] T ca81ae7de1fd444dbab0573bc43a5a84 P be696c8e15ee488e9915ed6330f3f614: Deleting consensus metadata
I20260809 04:40:43.039297 15328 ts_tablet_manager.cc:1939] T ca81ae7de1fd444dbab0573bc43a5a84 P 66897116c1df45759027f5fbf9c200f7: Deleting consensus metadata
I20260809 04:40:43.039330 14710 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:40:43.039824 14710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:40:43.039948 14693 catalog_manager.cc:4841] TS 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803): tablet ca81ae7de1fd444dbab0573bc43a5a84 (table test-table [id=1b5871ac21b942739809b1e1af6cad86]) successfully deleted
I20260809 04:40:43.039968 14696 catalog_manager.cc:4841] TS be696c8e15ee488e9915ed6330f3f614 (127.5.222.194:32899): tablet ca81ae7de1fd444dbab0573bc43a5a84 (table test-table [id=1b5871ac21b942739809b1e1af6cad86]) successfully deleted
W20260809 04:40:43.040436 14696 catalog_manager.cc:4808] TS be696c8e15ee488e9915ed6330f3f614 (127.5.222.194:32899): delete failed for tablet ca81ae7de1fd444dbab0573bc43a5a84 because the tablet was not found. No further retry: Not found: Tablet not found: ca81ae7de1fd444dbab0573bc43a5a84
I20260809 04:40:43.044234 14871 tablet_service.cc:1467] Processing CreateTablet for tablet cc0db65fb1c74a25b902f1c3082b4262 (DEFAULT_TABLE table=test-table [id=c04f69f857b8456d93691568d59b5686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:43.044394 15031 tablet_service.cc:1467] Processing CreateTablet for tablet cc0db65fb1c74a25b902f1c3082b4262 (DEFAULT_TABLE table=test-table [id=c04f69f857b8456d93691568d59b5686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:43.044525 14871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc0db65fb1c74a25b902f1c3082b4262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:43.044643 15031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc0db65fb1c74a25b902f1c3082b4262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:43.044654 15184 tablet_service.cc:1467] Processing CreateTablet for tablet cc0db65fb1c74a25b902f1c3082b4262 (DEFAULT_TABLE table=test-table [id=c04f69f857b8456d93691568d59b5686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:40:43.045116 15184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc0db65fb1c74a25b902f1c3082b4262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:43.047060 15298 tablet_bootstrap.cc:492] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7: Bootstrap starting.
I20260809 04:40:43.047351 15300 tablet_bootstrap.cc:492] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614: Bootstrap starting.
I20260809 04:40:43.047698 15299 tablet_bootstrap.cc:492] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7: Bootstrap starting.
I20260809 04:40:43.047974 15298 tablet_bootstrap.cc:654] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:43.048260 15300 tablet_bootstrap.cc:654] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:43.048588 15299 tablet_bootstrap.cc:654] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:43.049127 15298 tablet_bootstrap.cc:492] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7: No bootstrap required, opened a new log
I20260809 04:40:43.049252 15298 ts_tablet_manager.cc:1397] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 04:40:43.049687 15298 raft_consensus.cc:348] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:43.049799 15298 raft_consensus.cc:374] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:43.049834 15298 raft_consensus.cc:729] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66897116c1df45759027f5fbf9c200f7, State: Initialized, Role: FOLLOWER
I20260809 04:40:43.050005 15300 tablet_bootstrap.cc:492] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614: No bootstrap required, opened a new log
I20260809 04:40:43.049959 15298 consensus_queue.cc:260] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:43.050076 15300 ts_tablet_manager.cc:1397] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 04:40:43.050242 15298 ts_tablet_manager.cc:1428] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 04:40:43.050469 15300 raft_consensus.cc:348] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:43.050581 15300 raft_consensus.cc:374] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:43.050614 15300 raft_consensus.cc:729] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be696c8e15ee488e9915ed6330f3f614, State: Initialized, Role: FOLLOWER
I20260809 04:40:43.050738 15300 consensus_queue.cc:260] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:43.051044 15300 ts_tablet_manager.cc:1428] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 04:40:43.051452 15299 tablet_bootstrap.cc:492] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7: No bootstrap required, opened a new log
I20260809 04:40:43.051566 15299 ts_tablet_manager.cc:1397] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 04:40:43.051993 15299 raft_consensus.cc:348] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:43.052119 15299 raft_consensus.cc:374] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:43.052165 15299 raft_consensus.cc:729] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173a296a031a4a8cb49dfd26f3bd0ca7, State: Initialized, Role: FOLLOWER
I20260809 04:40:43.052289 15299 consensus_queue.cc:260] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:43.052558 15299 ts_tablet_manager.cc:1428] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 04:40:43.297708 15335 raft_consensus.cc:482] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:40:43.297922 15335 raft_consensus.cc:504] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:43.298514 15335 leader_election.cc:288] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341), 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803)
I20260809 04:40:43.299001 14895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc0db65fb1c74a25b902f1c3082b4262" candidate_uuid: "be696c8e15ee488e9915ed6330f3f614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" is_pre_election: true
I20260809 04:40:43.299022 15211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc0db65fb1c74a25b902f1c3082b4262" candidate_uuid: "be696c8e15ee488e9915ed6330f3f614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66897116c1df45759027f5fbf9c200f7" is_pre_election: true
I20260809 04:40:43.299185 14895 raft_consensus.cc:2454] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be696c8e15ee488e9915ed6330f3f614 in term 0.
I20260809 04:40:43.299175 15211 raft_consensus.cc:2454] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be696c8e15ee488e9915ed6330f3f614 in term 0.
I20260809 04:40:43.299505 14975 leader_election.cc:302] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66897116c1df45759027f5fbf9c200f7, be696c8e15ee488e9915ed6330f3f614; no voters:
I20260809 04:40:43.299733 15335 raft_consensus.cc:2781] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:40:43.299810 15335 raft_consensus.cc:482] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:40:43.299846 15335 raft_consensus.cc:3037] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:43.301090 15335 raft_consensus.cc:504] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:43.301445 15335 leader_election.cc:288] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [CANDIDATE]: Term 1 election: Requested vote from peers 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341), 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803)
I20260809 04:40:43.301793 14895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc0db65fb1c74a25b902f1c3082b4262" candidate_uuid: "be696c8e15ee488e9915ed6330f3f614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7"
I20260809 04:40:43.301856 15211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc0db65fb1c74a25b902f1c3082b4262" candidate_uuid: "be696c8e15ee488e9915ed6330f3f614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66897116c1df45759027f5fbf9c200f7"
I20260809 04:40:43.301946 14895 raft_consensus.cc:3037] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:43.301975 15211 raft_consensus.cc:3037] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:43.302901 14895 raft_consensus.cc:2454] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be696c8e15ee488e9915ed6330f3f614 in term 1.
I20260809 04:40:43.303050 15211 raft_consensus.cc:2454] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be696c8e15ee488e9915ed6330f3f614 in term 1.
I20260809 04:40:43.303189 14976 leader_election.cc:302] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173a296a031a4a8cb49dfd26f3bd0ca7, be696c8e15ee488e9915ed6330f3f614; no voters:
I20260809 04:40:43.303354 15335 raft_consensus.cc:2781] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:43.303449 15335 raft_consensus.cc:686] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 1 LEADER]: Becoming Leader. State: Replica: be696c8e15ee488e9915ed6330f3f614, State: Running, Role: LEADER
I20260809 04:40:43.303588 15335 consensus_queue.cc:237] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } }
I20260809 04:40:43.305351 14710 catalog_manager.cc:5495] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 reported cstate change: term changed from 0 to 1, leader changed from <none> to be696c8e15ee488e9915ed6330f3f614 (127.5.222.194). New cstate: current_term: 1 leader_uuid: "be696c8e15ee488e9915ed6330f3f614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be696c8e15ee488e9915ed6330f3f614" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 } health_report { overall_health: UNKNOWN } } }
I20260809 04:40:43.353281 15211 raft_consensus.cc:1261] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Refusing update from remote peer be696c8e15ee488e9915ed6330f3f614: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:43.353350 14895 raft_consensus.cc:1261] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Refusing update from remote peer be696c8e15ee488e9915ed6330f3f614: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:40:43.353793 15307 consensus_queue.cc:1035] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66897116c1df45759027f5fbf9c200f7" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:43.353986 15335 consensus_queue.cc:1035] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173a296a031a4a8cb49dfd26f3bd0ca7" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:40:43.384502 14710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:43.384701 14710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:40:43.387249 14710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c000cfc0a89429eb13779a5ec388655: Sending DeleteTablet for 3 replicas of tablet cc0db65fb1c74a25b902f1c3082b4262
I20260809 04:40:43.387948 14871 tablet_service.cc:1514] Processing DeleteTablet for tablet cc0db65fb1c74a25b902f1c3082b4262 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:43 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:33172
I20260809 04:40:43.387990 15031 tablet_service.cc:1514] Processing DeleteTablet for tablet cc0db65fb1c74a25b902f1c3082b4262 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:43 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:56406
I20260809 04:40:43.388022 15184 tablet_service.cc:1514] Processing DeleteTablet for tablet cc0db65fb1c74a25b902f1c3082b4262 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:40:43 UTC) from {username='oryx', principal='oryx/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:53336
I20260809 04:40:43.388134 15326 tablet_replica.cc:330] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7: stopping tablet replica
I20260809 04:40:43.388187 15325 tablet_replica.cc:330] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614: stopping tablet replica
I20260809 04:40:43.388258 15326 raft_consensus.cc:2229] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:43.388289 15325 raft_consensus.cc:2229] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:40:43.388365 15326 raft_consensus.cc:2258] T cc0db65fb1c74a25b902f1c3082b4262 P 173a296a031a4a8cb49dfd26f3bd0ca7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:43.388471 15325 raft_consensus.cc:2258] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:43.388646 15328 tablet_replica.cc:330] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7: stopping tablet replica
I20260809 04:40:43.388659 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 14795
I20260809 04:40:43.388793 15328 raft_consensus.cc:2229] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:40:43.388904 15328 raft_consensus.cc:2258] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:40:43.391462 15328 ts_tablet_manager.cc:1905] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:43.392812 15325 ts_tablet_manager.cc:1905] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:40:43.394196 15328 ts_tablet_manager.cc:1918] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:43.394289 15328 log.cc:1196] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/cc0db65fb1c74a25b902f1c3082b4262
I20260809 04:40:43.394569 15328 ts_tablet_manager.cc:1939] T cc0db65fb1c74a25b902f1c3082b4262 P 66897116c1df45759027f5fbf9c200f7: Deleting consensus metadata
I20260809 04:40:43.395484 15325 ts_tablet_manager.cc:1918] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:40:43.395483 14693 catalog_manager.cc:4841] TS 66897116c1df45759027f5fbf9c200f7 (127.5.222.195:42803): tablet cc0db65fb1c74a25b902f1c3082b4262 (table test-table [id=c04f69f857b8456d93691568d59b5686]) successfully deleted
I20260809 04:40:43.395581 15325 log.cc:1196] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/cc0db65fb1c74a25b902f1c3082b4262
I20260809 04:40:43.395828 15325 ts_tablet_manager.cc:1939] T cc0db65fb1c74a25b902f1c3082b4262 P be696c8e15ee488e9915ed6330f3f614: Deleting consensus metadata
I20260809 04:40:43.396464 14696 catalog_manager.cc:4841] TS be696c8e15ee488e9915ed6330f3f614 (127.5.222.194:32899): tablet cc0db65fb1c74a25b902f1c3082b4262 (table test-table [id=c04f69f857b8456d93691568d59b5686]) successfully deleted
W20260809 04:40:43.400110 14694 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:42341 (error 108)
I20260809 04:40:43.400172 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 14958
I20260809 04:40:43.403201 15402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:43.402818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:42341 (local address 127.0.0.1:33188)
0809 04:40:43.403021 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:43.403119 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":54,"threads_started":1}
W20260809 04:40:43.403339 14694 catalog_manager.cc:4584] TS 173a296a031a4a8cb49dfd26f3bd0ca7 (127.5.222.193:42341): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cc0db65fb1c74a25b902f1c3082b4262: Network error: Client connection negotiation failed: client connection to 127.5.222.193:42341: connect: Connection refused (error 111)
I20260809 04:40:43.408533 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 15115
I20260809 04:40:43.416347 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 14677
2026-08-09T04:40:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3551 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:43 dist-test-slave-vxj2 krb5kdc[15405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:43 dist-test-slave-vxj2 krb5kdc[15405](info): set up 2 sockets
Aug 09 04:40:43 dist-test-slave-vxj2 krb5kdc[15405](info): commencing operation
krb5kdc: starting...
W20260809 04:40:44.532359 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:44 dist-test-slave-vxj2 krb5kdc[15405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:40:44.655555 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:45371
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:34411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:45371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:44.764855 15420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:44.765097 15420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:44.765136 15420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:44.769188 15420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:44.769243 15420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:44.769261 15420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:44.769316 15420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:44.769348 15420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:44.774436 15420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:34411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:45371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:45371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:44.775435 15420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:44.776448 15420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:44.784791 15426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:44.784849 15420 server_base.cc:1034] running on GCE node
W20260809 04:40:44.784891 15427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:44.784791 15429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:44.785697 15420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:44.786762 15420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:44.787923 15420 hybrid_clock.cc:648] HybridClock initialized: now 1786250444787894 us; error 42 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T04:40:44Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1384 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:40:44 dist-test-slave-vxj2 krb5kdc[15434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:40:44 dist-test-slave-vxj2 krb5kdc[15434](info): set up 2 sockets
Aug 09 04:40:44 dist-test-slave-vxj2 krb5kdc[15434](info): commencing operation
krb5kdc: starting...
W20260809 04:40:45.937314 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:40:46 dist-test-slave-vxj2 krb5kdc[15434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:40:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:40:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:40:46.063486 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:37271
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:37271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:46.171797 15447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:46.172060 15447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:46.172098 15447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:46.176043 15447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:46.176101 15447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:46.176120 15447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:46.176138 15447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:46.176153 15447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:46.181257 15447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:37271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:37271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:46.182256 15447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:46.183213 15447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:46.191731 15457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:46.191797 15453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:46.191936 15447 server_base.cc:1034] running on GCE node
W20260809 04:40:46.191733 15459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:46.192664 15447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:46.193876 15447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:46.195042 15447 hybrid_clock.cc:648] HybridClock initialized: now 1786250446195025 us; error 31 us; skew 500 ppm
Aug 09 04:40:46 dist-test-slave-vxj2 krb5kdc[15434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250446, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:46.197788 15447 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:40:46.198670 15447 webserver.cc:460] Webserver started at http://127.5.222.254:34389/ using document root <none> and password file <none>
I20260809 04:40:46.199214 15447 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:46.199276 15447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:46.199494 15447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:46.201692 15447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "5789d8cca5ce4207b05726105b6da6c0"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "4bb46ec7ce8417a89fa218bb7a990191"
server_key_iv: "e45ff07cc7070d22b3604bbd8fc17294"
server_key_version: "encryptionkey@0"
I20260809 04:40:46.202258 15447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "5789d8cca5ce4207b05726105b6da6c0"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "4bb46ec7ce8417a89fa218bb7a990191"
server_key_iv: "e45ff07cc7070d22b3604bbd8fc17294"
server_key_version: "encryptionkey@0"
I20260809 04:40:46.206025 15447 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:40:46.208282 15467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:46.209647 15447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:46.209796 15447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "5789d8cca5ce4207b05726105b6da6c0"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "4bb46ec7ce8417a89fa218bb7a990191"
server_key_iv: "e45ff07cc7070d22b3604bbd8fc17294"
server_key_version: "encryptionkey@0"
I20260809 04:40:46.209915 15447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:46.243470 15447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:46.246593 15447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:46.246802 15447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:46.254541 15447 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:37271
I20260809 04:40:46.254570 15525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:37271 every 8 connection(s)
I20260809 04:40:46.255513 15447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:46.258293 15528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:40:46.259771 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 15447
I20260809 04:40:46.259915 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:40:46.260281 6011 external_mini_cluster.cc:1428] Setting key 619e44ede4ae3d82b588329150b32bbb
I20260809 04:40:46.264658 15528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0: Bootstrap starting.
I20260809 04:40:46.267022 15528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0: Neither blocks nor log segments found. Creating new log.
I20260809 04:40:46.267926 15528 log.cc:824] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:40:46 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:46.270236 15528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0: No bootstrap required, opened a new log
I20260809 04:40:46.275174 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:46.261822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48602 (local address 127.5.222.254:37271)
0809 04:40:46.262268 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:46.262283 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:46.262319 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:40:46.262939 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:46.262956 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:46.263266 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:46.263610 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:46.264447 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.265531 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.266421 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.266908 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.267086 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:46.269867 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:46.269875 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:46.269892 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:46.269924 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:46.271928 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:46.272372 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:46.272379 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:46.272387 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:46.272557 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:46.272850 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:46.272854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:46.272859 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:46.273437 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:46.273629 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:46.273878 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 04:40:46.274179 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":128,"threads_started":1}
I20260809 04:40:46.275146 15528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5789d8cca5ce4207b05726105b6da6c0" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37271 } }
I20260809 04:40:46.275444 15528 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:40:46.275504 15528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5789d8cca5ce4207b05726105b6da6c0, State: Initialized, Role: FOLLOWER
I20260809 04:40:46.275988 15528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5789d8cca5ce4207b05726105b6da6c0" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37271 } }
I20260809 04:40:46.276157 15528 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:46.276225 15528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:46.276329 15528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:40:46.277351 15528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5789d8cca5ce4207b05726105b6da6c0" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37271 } }
I20260809 04:40:46.277722 15528 leader_election.cc:302] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5789d8cca5ce4207b05726105b6da6c0; no voters:
I20260809 04:40:46.278044 15528 leader_election.cc:288] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:40:46.278193 15533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:40:46.278440 15533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 1 LEADER]: Becoming Leader. State: Replica: 5789d8cca5ce4207b05726105b6da6c0, State: Running, Role: LEADER
I20260809 04:40:46.278792 15533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5789d8cca5ce4207b05726105b6da6c0" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37271 } }
I20260809 04:40:46.279228 15528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:46.280534 15534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5789d8cca5ce4207b05726105b6da6c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5789d8cca5ce4207b05726105b6da6c0" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37271 } } }
I20260809 04:40:46.280680 15534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:46.281105 15542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:46.281073 15535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5789d8cca5ce4207b05726105b6da6c0. Latest consensus state: current_term: 1 leader_uuid: "5789d8cca5ce4207b05726105b6da6c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5789d8cca5ce4207b05726105b6da6c0" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37271 } } }
I20260809 04:40:46.281302 15535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:46.284358 15542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:46.290177 15542 catalog_manager.cc:1347] Generated new cluster ID: 12aa72fca3b74e6bb9fe245664055846
I20260809 04:40:46.290247 15542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:46.300319 15542 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:40:46.301352 15542 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:46.310107 15542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0: Generated new TSK 0
I20260809 04:40:46.310828 15542 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:40:46.349218 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37271
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:46.459414 15556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:46.459647 15556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:46.459692 15556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:46.463617 15556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:46.463666 15556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:46.463748 15556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:46.468847 15556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:46.469837 15556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:46.470844 15556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:40:46.480672 15556 server_base.cc:1034] running on GCE node
W20260809 04:40:46.480621 15566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:46.480640 15563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:46.480605 15562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:46.481428 15556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:46.482103 15556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:46.483282 15556 hybrid_clock.cc:648] HybridClock initialized: now 1786250446483244 us; error 50 us; skew 500 ppm
Aug 09 04:40:46 dist-test-slave-vxj2 krb5kdc[15434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250446, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:46.486166 15556 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:40:46.487030 15556 webserver.cc:460] Webserver started at http://127.5.222.193:43617/ using document root <none> and password file <none>
I20260809 04:40:46.487584 15556 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:46.487676 15556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:46.487946 15556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:46.490032 15556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "a382fbcc1f324c08ab6ccadaff6d5fc3"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "77f61fb85ad15e8537dc811b64ccfc9a"
server_key_iv: "55954545e18ec8bd0a04155c3b079605"
server_key_version: "encryptionkey@0"
I20260809 04:40:46.490609 15556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "a382fbcc1f324c08ab6ccadaff6d5fc3"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "77f61fb85ad15e8537dc811b64ccfc9a"
server_key_iv: "55954545e18ec8bd0a04155c3b079605"
server_key_version: "encryptionkey@0"
I20260809 04:40:46.494391 15556 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:40:46.496644 15573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:46.497829 15556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:46.497963 15556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "a382fbcc1f324c08ab6ccadaff6d5fc3"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "77f61fb85ad15e8537dc811b64ccfc9a"
server_key_iv: "55954545e18ec8bd0a04155c3b079605"
server_key_version: "encryptionkey@0"
I20260809 04:40:46.498070 15556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:46.516777 15556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:46.520251 15556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:46.520496 15556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:46.521129 15556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:46.522015 15556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:46.522094 15556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:46.522176 15556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:46.522233 15556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:46.532347 15556 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:43271
I20260809 04:40:46.532369 15700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:43271 every 8 connection(s)
I20260809 04:40:46.533439 15556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:46.535671 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 15556
I20260809 04:40:46.535815 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:40:46.536185 6011 external_mini_cluster.cc:1428] Setting key 5ddc359270fb74af1df6ab314ee6d6b0
Aug 09 04:40:46 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250446, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:40:46.548637 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:46.535339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:49745 (local address 127.5.222.254:37271)
0809 04:40:46.535484 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:46.535489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:46.536486 (+ 997us) server_negotiation.cc:407] Connection header received
0809 04:40:46.538152 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:46.538156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:46.538228 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:46.538316 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:46.539493 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.540054 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.541318 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.541683 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.541814 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:46.544620 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:46.544624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:46.544626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:46.544652 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:46.546371 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:46.546952 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:46.546957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:46.546959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:46.547055 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:46.547441 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:46.547445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:46.547447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:46.547645 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:46.547773 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:46.548400 (+ 627us) server_negotiation.cc:299] Negotiation successful
0809 04:40:46.548514 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:40:46.549836 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:46.535772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:49745)
0809 04:40:46.536337 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:46.536382 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:46.537933 (+ 1551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:46.538494 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:46.538501 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:46.538863 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:46.539338 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:46.539352 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.540197 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.540201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:46.541093 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:46.541102 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.541851 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.541856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:46.542048 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:46.542671 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:46.542696 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:46.544437 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:46.546536 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:46.546542 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:46.546557 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:46.546837 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:46.547159 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:46.547164 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:46.547166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:46.547325 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:46.547783 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:46.547793 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:46.548088 (+ 295us) client_negotiation.cc:769] Sending connection context
0809 04:40:46.548329 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 04:40:46.548713 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":156,"threads_started":1}
I20260809 04:40:46.551128 15701 heartbeater.cc:347] Connected to a master server at 127.5.222.254:37271
I20260809 04:40:46.551390 15701 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:46.551896 15701 heartbeater.cc:511] Master 127.5.222.254:37271 requested a full tablet report, sending...
I20260809 04:40:46.553509 15488 ts_manager.cc:194] Registered new tserver with Master: a382fbcc1f324c08ab6ccadaff6d5fc3 (127.5.222.193:43271)
I20260809 04:40:46.554679 15488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:49745
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:40:46.563979 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37271
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:46.676131 15706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:46.676393 15706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:46.676442 15706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:46.680740 15706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:46.680828 15706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:46.680943 15706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:46.686311 15706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:46.687443 15706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:46.688458 15706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:46.698042 15716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:46.698051 15712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:46.698067 15714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:46.698447 15706 server_base.cc:1034] running on GCE node
I20260809 04:40:46.698900 15706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:46.699530 15706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:46.700723 15706 hybrid_clock.cc:648] HybridClock initialized: now 1786250446700703 us; error 28 us; skew 500 ppm
Aug 09 04:40:46 dist-test-slave-vxj2 krb5kdc[15434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250446, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:46.703671 15706 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:40:46.704598 15706 webserver.cc:460] Webserver started at http://127.5.222.194:34573/ using document root <none> and password file <none>
I20260809 04:40:46.705227 15706 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:46.705331 15706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:46.705571 15706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:46.707674 15706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "3ccd794b810044a1bd8933a2b5dcbd63"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "a3e401dad57fda3d86aba093afdc919b"
server_key_iv: "770ca73f2f46e1ced833ee892be8608b"
server_key_version: "encryptionkey@0"
I20260809 04:40:46.708336 15706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "3ccd794b810044a1bd8933a2b5dcbd63"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "a3e401dad57fda3d86aba093afdc919b"
server_key_iv: "770ca73f2f46e1ced833ee892be8608b"
server_key_version: "encryptionkey@0"
I20260809 04:40:46.712188 15706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:40:46.714673 15724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:46.715907 15706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:40:46.716051 15706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "3ccd794b810044a1bd8933a2b5dcbd63"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "a3e401dad57fda3d86aba093afdc919b"
server_key_iv: "770ca73f2f46e1ced833ee892be8608b"
server_key_version: "encryptionkey@0"
I20260809 04:40:46.716209 15706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:46.744626 15706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:46.747555 15706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:46.747778 15706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:46.748464 15706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:46.749363 15706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:46.749433 15706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:46.749506 15706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:46.749550 15706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:46.758214 15706 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:35013
I20260809 04:40:46.758260 15851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:35013 every 8 connection(s)
I20260809 04:40:46.759078 15706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:46.760658 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 15706
I20260809 04:40:46.760816 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:40:46.761134 6011 external_mini_cluster.cc:1428] Setting key 89ce2bf0ff55f017ac818ab985f6bbb1
Aug 09 04:40:46 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250446, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
I20260809 04:40:46.775647 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:46.760841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:57515 (local address 127.5.222.254:37271)
0809 04:40:46.760973 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:46.760978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:46.762048 (+ 1070us) server_negotiation.cc:407] Connection header received
0809 04:40:46.763514 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:46.763520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:46.763580 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:46.763672 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:46.764842 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.765757 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.768012 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.768267 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.768361 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:46.771455 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:46.771461 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:46.771464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:46.771497 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:46.773098 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:46.773806 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:46.773809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:46.773811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:46.773891 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:46.774308 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:46.774315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:46.774317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:46.774538 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:46.774668 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:46.775293 (+ 625us) server_negotiation.cc:299] Negotiation successful
0809 04:40:46.775436 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 04:40:46.776502 15854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:46.761320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:57515)
0809 04:40:46.761872 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:46.761929 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:46.763304 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:46.763819 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:46.763826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:46.764161 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:46.764691 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:46.764705 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.765939 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.765947 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:46.767802 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:46.767820 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.768454 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.768461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:46.768716 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:46.769448 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:46.769468 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:46.771262 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:46.773270 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:46.773275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:46.773287 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:46.773668 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:46.773990 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:46.773992 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:46.773994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:46.774167 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:46.774680 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:46.774742 (+ 62us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:46.774991 (+ 249us) client_negotiation.cc:769] Sending connection context
0809 04:40:46.775216 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 04:40:46.775578 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":139,"threads_started":1}
I20260809 04:40:46.777918 15852 heartbeater.cc:347] Connected to a master server at 127.5.222.254:37271
I20260809 04:40:46.778172 15852 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:46.778697 15852 heartbeater.cc:511] Master 127.5.222.254:37271 requested a full tablet report, sending...
I20260809 04:40:46.779765 15488 ts_manager.cc:194] Registered new tserver with Master: 3ccd794b810044a1bd8933a2b5dcbd63 (127.5.222.194:35013)
I20260809 04:40:46.780572 15488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:57515
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:40:46.792516 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37271
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:46.903843 15860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:46.904119 15860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:46.904196 15860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:46.908183 15860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:46.908272 15860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:46.908361 15860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:46.913648 15860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:37271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:46.914741 15860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:46.915776 15860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:46.925390 15867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:46.925482 15871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:46.925467 15860 server_base.cc:1034] running on GCE node
W20260809 04:40:46.925402 15869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:46.926184 15860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:46.926812 15860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:46.927999 15860 hybrid_clock.cc:648] HybridClock initialized: now 1786250446927968 us; error 44 us; skew 500 ppm
Aug 09 04:40:46 dist-test-slave-vxj2 krb5kdc[15434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250446, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:46.930711 15860 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:40:46.931591 15860 webserver.cc:460] Webserver started at http://127.5.222.195:35873/ using document root <none> and password file <none>
I20260809 04:40:46.932143 15860 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:46.932237 15860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:46.932467 15860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:40:46.934508 15860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "df38ff5cd14049909dc8ff9af1f5ccf0"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "4b8347ed8019d9d53e2d3701992659e3"
server_key_iv: "326759c74c44af1fdd63d6e0b7a7a404"
server_key_version: "encryptionkey@0"
I20260809 04:40:46.935070 15860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "df38ff5cd14049909dc8ff9af1f5ccf0"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "4b8347ed8019d9d53e2d3701992659e3"
server_key_iv: "326759c74c44af1fdd63d6e0b7a7a404"
server_key_version: "encryptionkey@0"
I20260809 04:40:46.938669 15860 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:40:46.940944 15880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:46.942199 15860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:46.942337 15860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "df38ff5cd14049909dc8ff9af1f5ccf0"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "4b8347ed8019d9d53e2d3701992659e3"
server_key_iv: "326759c74c44af1fdd63d6e0b7a7a404"
server_key_version: "encryptionkey@0"
I20260809 04:40:46.942444 15860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:46.961208 15860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:46.963979 15860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:46.964169 15860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:46.964736 15860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:46.965603 15860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:46.965677 15860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:46.965746 15860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:46.965795 15860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:46.974615 15860 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:35657
I20260809 04:40:46.974638 16013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:35657 every 8 connection(s)
I20260809 04:40:46.975452 15860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:40:46.978894 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 15860
I20260809 04:40:46.979056 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:40:46.979390 6011 external_mini_cluster.cc:1428] Setting key 61a96dc7aa33f3ff14071d2bb30c73c9
Aug 09 04:40:46 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250446, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:40:46.989666 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:46.977303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:42229 (local address 127.5.222.254:37271)
0809 04:40:46.977442 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:46.977446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:46.978216 (+ 770us) server_negotiation.cc:407] Connection header received
0809 04:40:46.979915 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:46.979918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:46.980001 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:46.980104 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:46.981483 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.982014 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.983193 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.983444 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.983556 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:46.986052 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:46.986055 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:46.986058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:46.986091 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:46.987516 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:46.988087 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:46.988089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:46.988091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:46.988164 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:46.988542 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:46.988545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:46.988546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:46.988767 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:46.988872 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:46.989444 (+ 572us) server_negotiation.cc:299] Negotiation successful
0809 04:40:46.989548 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:40:46.990477 16016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:46.977649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:42229)
0809 04:40:46.978074 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:46.978114 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:46.979660 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:46.980292 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:46.980306 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:46.980772 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:46.981348 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:46.981363 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.982139 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.982149 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:46.983004 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:46.983013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:46.983593 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:46.983597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:46.983768 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:46.984362 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:46.984378 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:46.985892 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:40:46.987645 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:46.987651 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:46.987664 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:46.987927 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:46.988259 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:40:46.988263 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:40:46.988265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:40:46.988406 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:40:46.988905 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:40:46.988911 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:40:46.989134 (+ 223us) client_negotiation.cc:769] Sending connection context
0809 04:40:46.989371 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 04:40:46.989638 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":114,"threads_started":1}
I20260809 04:40:46.991575 16014 heartbeater.cc:347] Connected to a master server at 127.5.222.254:37271
I20260809 04:40:46.991775 16014 heartbeater.cc:464] Registering TS with master...
I20260809 04:40:46.992264 16014 heartbeater.cc:511] Master 127.5.222.254:37271 requested a full tablet report, sending...
I20260809 04:40:46.993395 15487 ts_manager.cc:194] Registered new tserver with Master: df38ff5cd14049909dc8ff9af1f5ccf0 (127.5.222.195:35657)
I20260809 04:40:46.994048 15487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:42229
I20260809 04:40:47.005223 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab.
Entry for principal oryx/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab.
WARNING: no policy specified for oryx/127.5.222.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab.
Entry for principal oryx/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab.
WARNING: no policy specified for oryx/127.5.222.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.222.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab.
Entry for principal oryx/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab.
I20260809 04:40:47.082098 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 15556
I20260809 04:40:47.091380 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 15706
I20260809 04:40:47.098037 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 15860
I20260809 04:40:47.105263 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 15447
I20260809 04:40:47.114063 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:37271
--webserver_interface=127.5.222.254
--webserver_port=34389
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:37271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:40:47.225618 16024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:47.225914 16024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:47.226020 16024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:47.230037 16024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:40:47.230118 16024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:47.230154 16024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:47.230209 16024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:40:47.230247 16024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:40:47.235507 16024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:37271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:37271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=34389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:47.236542 16024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:47.237630 16024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:47.246479 16030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:47.246479 16029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:47.246569 16032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:47.246672 16024 server_base.cc:1034] running on GCE node
I20260809 04:40:47.247320 16024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:47.248356 16024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:47.249537 16024 hybrid_clock.cc:648] HybridClock initialized: now 1786250447249506 us; error 42 us; skew 500 ppm
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250447, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:47.252578 16024 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:40:47.253551 16024 webserver.cc:460] Webserver started at http://127.5.222.254:34389/ using document root <none> and password file <none>
I20260809 04:40:47.254096 16024 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:47.254169 16024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:47.257982 16024 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:47.260253 16040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:47.261584 16024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:40:47.261729 16024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "5789d8cca5ce4207b05726105b6da6c0"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "4bb46ec7ce8417a89fa218bb7a990191"
server_key_iv: "e45ff07cc7070d22b3604bbd8fc17294"
server_key_version: "encryptionkey@0"
I20260809 04:40:47.262303 16024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:47.273016 16024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:47.276083 16024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:47.276288 16024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:47.283771 16024 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:37271
I20260809 04:40:47.283806 16102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:37271 every 8 connection(s)
I20260809 04:40:47.284792 16024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:40:47.288899 16103 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:40:47.290504 16103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0: Bootstrap starting.
I20260809 04:40:47.290838 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 16024
I20260809 04:40:47.291353 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:43271
--local_ip_for_outbound_sockets=127.5.222.193
--tserver_master_addrs=127.5.222.254:37271
--webserver_port=43617
--webserver_interface=127.5.222.193
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.222.193
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:40:47.308240 16103 log.cc:824] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0: Log is configured to *not* fsync() on all Append() calls
I20260809 04:40:47.314888 16103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:40:47.315279 16103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0: Bootstrap complete.
I20260809 04:40:47.319608 16103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5789d8cca5ce4207b05726105b6da6c0" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37271 } }
I20260809 04:40:47.320068 16103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5789d8cca5ce4207b05726105b6da6c0, State: Initialized, Role: FOLLOWER
I20260809 04:40:47.320531 16103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5789d8cca5ce4207b05726105b6da6c0" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37271 } }
I20260809 04:40:47.320674 16103 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:40:47.320765 16103 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:40:47.320883 16103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:40:47.322862 16103 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5789d8cca5ce4207b05726105b6da6c0" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37271 } }
I20260809 04:40:47.323189 16103 leader_election.cc:302] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5789d8cca5ce4207b05726105b6da6c0; no voters:
I20260809 04:40:47.323498 16103 leader_election.cc:288] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:40:47.323634 16107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:40:47.323909 16107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [term 2 LEADER]: Becoming Leader. State: Replica: 5789d8cca5ce4207b05726105b6da6c0, State: Running, Role: LEADER
I20260809 04:40:47.324281 16107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5789d8cca5ce4207b05726105b6da6c0" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37271 } }
I20260809 04:40:47.324688 16103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:40:47.325757 16109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5789d8cca5ce4207b05726105b6da6c0. Latest consensus state: current_term: 2 leader_uuid: "5789d8cca5ce4207b05726105b6da6c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5789d8cca5ce4207b05726105b6da6c0" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37271 } } }
I20260809 04:40:47.325883 16109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:47.326236 16108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5789d8cca5ce4207b05726105b6da6c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5789d8cca5ce4207b05726105b6da6c0" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 37271 } } }
I20260809 04:40:47.326321 16108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0 [sys.catalog]: This master's current role is: LEADER
I20260809 04:40:47.327167 16118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:40:47.330255 16118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:40:47.330876 16118 catalog_manager.cc:1259] Loaded cluster ID: 12aa72fca3b74e6bb9fe245664055846
I20260809 04:40:47.330937 16118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:40:47.333283 16118 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:40:47.333918 16118 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5789d8cca5ce4207b05726105b6da6c0: Loaded TSK: 0
I20260809 04:40:47.335068 16118 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:40:47.408097 16105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:47.408375 16105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:47.408448 16105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:47.412490 16105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:47.412593 16105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:47.412693 16105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:40:47.417927 16105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.193.keytab
--principal=oryx/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:43271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=43617
--tserver_master_addrs=127.5.222.254:37271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:47.418929 16105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:47.419915 16105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:47.430773 16133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:47.430757 16138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:47.430763 16135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:47.431280 16105 server_base.cc:1034] running on GCE node
I20260809 04:40:47.431653 16105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:47.432317 16105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:47.433491 16105 hybrid_clock.cc:648] HybridClock initialized: now 1786250447433479 us; error 29 us; skew 500 ppm
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250447, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:47.436234 16105 init.cc:377] Logged in from keytab as oryx/127.5.222.193@KRBTEST.COM (short username oryx)
I20260809 04:40:47.437212 16105 webserver.cc:460] Webserver started at http://127.5.222.193:43617/ using document root <none> and password file <none>
I20260809 04:40:47.437774 16105 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:47.437848 16105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:47.441774 16105 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:40:47.444010 16147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:47.445312 16105 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:40:47.445439 16105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "a382fbcc1f324c08ab6ccadaff6d5fc3"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "77f61fb85ad15e8537dc811b64ccfc9a"
server_key_iv: "55954545e18ec8bd0a04155c3b079605"
server_key_version: "encryptionkey@0"
I20260809 04:40:47.445959 16105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:47.459964 16105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:47.463119 16105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:47.463317 16105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:47.463949 16105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:47.464846 16105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:47.464895 16105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:47.464967 16105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:47.465001 16105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:47.475826 16105 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:43271
I20260809 04:40:47.475857 16279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:43271 every 8 connection(s)
I20260809 04:40:47.476831 16105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:40:47.478243 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 16105
I20260809 04:40:47.478760 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:35013
--local_ip_for_outbound_sockets=127.5.222.194
--tserver_master_addrs=127.5.222.254:37271
--webserver_port=34573
--webserver_interface=127.5.222.194
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.222.194
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:47.490464 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:47.479331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:51321)
0809 04:40:47.479813 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:47.479867 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:47.481079 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:47.482242 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:47.482250 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:47.482673 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:47.483143 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.483157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.484426 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.484431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.485363 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.485373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.486112 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.486116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.486301 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.486999 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:47.487015 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:47.489499 (+ 2484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":145,"threads_started":1}
W20260809 04:40:47.490453 16284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:47.479382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:51321 (local address 127.5.222.254:37271)
0809 04:40:47.480053 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:47.480075 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:47.480137 (+ 62us) server_negotiation.cc:407] Connection header received
0809 04:40:47.481551 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:47.481562 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:47.481843 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:47.482172 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:47.483329 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.484279 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.485489 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.485946 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.486121 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.489711 (+ 3590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:51321: BlockingRecv error: recv got EOF from 127.5.222.193:51321 (error 108)
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":140,"threads_started":1}
W20260809 04:40:47.491438 16281 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:37271 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.222.254:37271: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:47.496855 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:47.491922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:57403)
0809 04:40:47.492018 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:47.492034 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:47.492116 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:47.492359 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:47.492362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:47.492517 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:47.492603 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.492608 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.493371 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.493376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.493905 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.493921 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.494364 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.494367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.494459 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.494983 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:47.494994 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:47.496717 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 04:40:47.496953 16284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:47.492020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:57403 (local address 127.5.222.254:37271)
0809 04:40:47.492165 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:47.492169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:47.492184 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:40:47.492224 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:47.492226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:47.492263 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:47.492346 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:47.492716 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.493268 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.494040 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.494256 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.494361 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.496838 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:57403: BlockingRecv error: recv got EOF from 127.5.222.193:57403 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:47.502985 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:47.497596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:45243)
0809 04:40:47.497737 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:47.497752 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:47.497837 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:47.498071 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:47.498074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:47.498245 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:47.498338 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.498344 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.499012 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.499015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.499983 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.499994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.500398 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.500401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.500490 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.500919 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:47.500934 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:47.502856 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 04:40:47.503083 16284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:47.497705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:45243 (local address 127.5.222.254:37271)
0809 04:40:47.497818 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:47.497821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:47.497833 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:40:47.497927 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:47.497929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:47.497962 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:47.498026 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:47.498452 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.498902 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.500096 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.500290 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.500362 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.502972 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:45243: BlockingRecv error: recv got EOF from 127.5.222.193:45243 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 04:40:47.503201 16281 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:40:47.605651 16285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:47.605952 16285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:47.605996 16285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:47.610145 16285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:47.610215 16285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:47.610323 16285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:40:47.615823 16285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.194.keytab
--principal=oryx/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:35013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=34573
--tserver_master_addrs=127.5.222.254:37271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:47.617071 16285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:47.618139 16285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:47.627718 16292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:47.627724 16294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:47.627802 16291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:47.628062 16285 server_base.cc:1034] running on GCE node
I20260809 04:40:47.628516 16285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:47.629263 16285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:47.630450 16285 hybrid_clock.cc:648] HybridClock initialized: now 1786250447630428 us; error 38 us; skew 500 ppm
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250447, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:47.633121 16285 init.cc:377] Logged in from keytab as oryx/127.5.222.194@KRBTEST.COM (short username oryx)
I20260809 04:40:47.634088 16285 webserver.cc:460] Webserver started at http://127.5.222.194:34573/ using document root <none> and password file <none>
I20260809 04:40:47.634651 16285 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:47.634761 16285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:47.638540 16285 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 04:40:47.640524 16302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:47.641747 16285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:47.641841 16285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "3ccd794b810044a1bd8933a2b5dcbd63"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "a3e401dad57fda3d86aba093afdc919b"
server_key_iv: "770ca73f2f46e1ced833ee892be8608b"
server_key_version: "encryptionkey@0"
I20260809 04:40:47.642330 16285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:47.673703 16285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:47.677016 16285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:47.677191 16285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:47.677749 16285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:47.678534 16285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:47.678581 16285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:47.678620 16285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:47.678637 16285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:47.687976 16285 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:35013
I20260809 04:40:47.688045 16434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:35013 every 8 connection(s)
I20260809 04:40:47.689009 16285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:40:47.696861 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 16285
I20260809 04:40:47.697419 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:35657
--local_ip_for_outbound_sockets=127.5.222.195
--tserver_master_addrs=127.5.222.254:37271
--webserver_port=35873
--webserver_interface=127.5.222.195
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.222.195
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:40:47.700450 16284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:47.691031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:39861 (local address 127.5.222.254:37271)
0809 04:40:47.691173 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:47.691176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:47.692021 (+ 845us) server_negotiation.cc:407] Connection header received
0809 04:40:47.693309 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:47.693312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:47.693363 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:47.693448 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:47.694594 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.695123 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.696197 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.696436 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.696572 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.700304 (+ 3732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:39861: BlockingRecv error: recv got EOF from 127.5.222.194:39861 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:40:47.701118 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:47.691421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:39861)
0809 04:40:47.691849 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:47.691890 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:47.693048 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:47.693605 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:47.693611 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:47.693972 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:47.694428 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.694440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.695230 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.695233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.696081 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.696089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.696575 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.696578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.696745 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.697592 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:47.697611 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:47.700117 (+ 2506us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":117,"threads_started":1}
W20260809 04:40:47.701624 16435 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:37271 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.222.254:37271: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:47.707020 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:47.702079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:39845)
0809 04:40:47.702172 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:47.702187 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:47.702262 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:47.702504 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:47.702507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:47.702672 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:47.702754 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.702760 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.703480 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.703485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.704028 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.704036 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.704485 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.704489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.704578 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.705078 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:47.705098 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:47.706885 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 04:40:47.707120 16284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:47.702166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:39845 (local address 127.5.222.254:37271)
0809 04:40:47.702286 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:47.702293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:47.702308 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:40:47.702351 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:47.702354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:47.702400 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:47.702490 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:47.702868 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.703369 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.704150 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.704375 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.704484 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.707009 (+ 2525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:39845: BlockingRecv error: recv got EOF from 127.5.222.194:39845 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:47.712366 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:47.707580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:49761)
0809 04:40:47.707667 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:47.707683 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:47.707753 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:47.707988 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:47.707990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:47.708137 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:47.708199 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.708204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.708899 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.708905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.709499 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.709508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.709966 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.709969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.710052 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.710504 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:47.710516 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:47.712251 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 04:40:47.712576 16284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:47.707673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:49761 (local address 127.5.222.254:37271)
0809 04:40:47.707783 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:47.707790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:47.707805 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:40:47.707840 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:47.707842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:47.707883 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:47.707974 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:47.708309 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.708793 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.709617 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.709840 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.709962 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.712353 (+ 2391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:49761: BlockingRecv error: recv got EOF from 127.5.222.194:49761 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 04:40:47.712795 16435 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:40:47.810715 16439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:40:47.811041 16439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:40:47.811112 16439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:40:47.815227 16439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:40:47.815320 16439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:40:47.815413 16439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:40:47.820822 16439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/krb5kdc/oryx_127.5.222.195.keytab
--principal=oryx/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:35657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=35873
--tserver_master_addrs=127.5.222.254:37271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:40:47.821973 16439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:40:47.823061 16439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:40:47.832969 16447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:47.833010 16450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:40:47.833010 16446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:40:47.833293 16439 server_base.cc:1034] running on GCE node
I20260809 04:40:47.833678 16439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:40:47.834391 16439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:40:47.835573 16439 hybrid_clock.cc:648] HybridClock initialized: now 1786250447835540 us; error 45 us; skew 500 ppm
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250447, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:40:47.838303 16439 init.cc:377] Logged in from keytab as oryx/127.5.222.195@KRBTEST.COM (short username oryx)
I20260809 04:40:47.839128 16439 webserver.cc:460] Webserver started at http://127.5.222.195:35873/ using document root <none> and password file <none>
I20260809 04:40:47.839690 16439 fs_manager.cc:359] Metadata directory not provided
I20260809 04:40:47.839803 16439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:40:47.844024 16439 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:40:47.846094 16457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:47.847348 16439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:40:47.847445 16439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "df38ff5cd14049909dc8ff9af1f5ccf0"
format_stamp: "Formatted at 2026-08-09 04:40:46 on dist-test-slave-vxj2"
server_key: "4b8347ed8019d9d53e2d3701992659e3"
server_key_iv: "326759c74c44af1fdd63d6e0b7a7a404"
server_key_version: "encryptionkey@0"
I20260809 04:40:47.847947 16439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:40:47.871058 16439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:40:47.873978 16439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:40:47.874169 16439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:40:47.874807 16439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:40:47.875669 16439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:40:47.875718 16439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:47.875789 16439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:40:47.875829 16439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:40:47.885154 16439 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:35657
I20260809 04:40:47.885212 16584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:35657 every 8 connection(s)
I20260809 04:40:47.886197 16439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:47.895433 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 16439
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:40:47.897264 16284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:47.887968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:38767 (local address 127.5.222.254:37271)
0809 04:40:47.888118 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:47.888122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:47.888916 (+ 794us) server_negotiation.cc:407] Connection header received
0809 04:40:47.890241 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:47.890244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:47.890301 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:47.890423 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:47.891538 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.892057 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.893205 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.893434 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.893547 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.897098 (+ 3551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:38767: BlockingRecv error: recv got EOF from 127.5.222.195:38767 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:40:47.897926 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:47.888338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:38767)
0809 04:40:47.888764 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:47.888805 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:47.890027 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:47.890583 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:47.890590 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:47.890944 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:47.891389 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.891402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.892180 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.892191 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.893055 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.893065 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.893585 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.893589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.893751 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.894347 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:47.894362 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:47.896992 (+ 2630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":117,"threads_started":1}
W20260809 04:40:47.898469 16585 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:37271 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.222.254:37271: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:47.903666 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:47.898859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:37663)
0809 04:40:47.898963 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:47.898977 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:47.899050 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:47.899301 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:47.899303 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:47.899449 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:47.899532 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.899537 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.900246 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.900250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.900790 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.900798 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.901294 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.901298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.901399 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.901843 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:47.901853 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:47.903533 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 04:40:47.903756 16284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:47.898970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:37663 (local address 127.5.222.254:37271)
0809 04:40:47.899077 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:47.899083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:47.899110 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:40:47.899144 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:47.899147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:47.899186 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:47.899272 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:47.899646 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.900138 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.900912 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.901129 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.901290 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.903642 (+ 2352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:37663: BlockingRecv error: recv got EOF from 127.5.222.195:37663 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 04:40:47.906180 16589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:47.896421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48614 (local address 127.5.222.254:37271)
0809 04:40:47.896642 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:47.896646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:47.897480 (+ 834us) server_negotiation.cc:407] Connection header received
0809 04:40:47.897685 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:47.897688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:47.897737 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:47.897829 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:47.898638 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.899267 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.900112 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.900343 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.900464 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.901991 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:40:47.901997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:40:47.902021 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:40:47.902046 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:40:47.903891 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:47.904404 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:47.904411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:47.904415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:47.904584 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:40:47.904873 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:40:47.904877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:40:47.904878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:40:47.905308 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:40:47.905520 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 04:40:47.905818 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 04:40:47.906069 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:47 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:47.909647 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:47.904228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:36027)
0809 04:40:47.904333 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:47.904349 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:47.904431 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:47.904766 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:47.904769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:47.904907 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:47.904970 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.904975 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.905736 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.905748 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.906467 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:47.906480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.906994 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.906999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:47.907106 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.907561 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:47.907575 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:47.909514 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 04:40:47.909724 16284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:47.904409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:36027 (local address 127.5.222.254:37271)
0809 04:40:47.904570 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:47.904574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:47.904587 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:40:47.904619 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:47.904624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:47.904670 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:47.904739 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:47.905084 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.905630 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.906592 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:47.906875 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:47.906987 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:47.909614 (+ 2627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:36027: BlockingRecv error: recv got EOF from 127.5.222.195:36027 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260809 04:40:47.909919 16585 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 04:40:48 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:48 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:48.509595 16590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:48.503803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:46051)
0809 04:40:48.504109 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:48.504122 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:48.504216 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:48.504492 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:48.504495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:48.504679 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:48.504821 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:48.504828 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:48.505656 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:48.505659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:48.506310 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:48.506320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:48.506838 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:48.506841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:48.506911 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:48.507417 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:48.507429 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:48.509468 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":90,"threads_started":1}
W20260809 04:40:48.509760 16592 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:48.503953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:46051 (local address 127.5.222.254:37271)
0809 04:40:48.504205 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:48.504208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:48.504221 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:40:48.504285 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:48.504287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:48.504328 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:48.504423 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:48.504936 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:48.505548 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:48.506445 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:48.506683 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:48.506763 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:48.509612 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:46051: BlockingRecv error: recv got EOF from 127.5.222.193:46051 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":108,"threads_started":1}
Aug 09 04:40:48 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:48 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:48.718911 16593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:48.713464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:35093)
0809 04:40:48.713776 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:48.713791 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:48.713892 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:48.714113 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:48.714115 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:48.714301 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:48.714408 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:48.714414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:48.715143 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:48.715147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:48.715727 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:48.715735 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:48.716176 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:48.716179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:48.716245 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:48.716715 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:48.716727 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:48.718781 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":89,"threads_started":1}
W20260809 04:40:48.719019 16592 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:48.713509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:35093 (local address 127.5.222.254:37271)
0809 04:40:48.713654 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:48.713658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:48.713861 (+ 203us) server_negotiation.cc:407] Connection header received
0809 04:40:48.713969 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:48.713971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:48.714016 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:48.714076 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:48.714512 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:48.715033 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:48.715870 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:48.716081 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:48.716154 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:48.718886 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:35093: BlockingRecv error: recv got EOF from 127.5.222.194:35093 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 04:40:48 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:48 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:48.915908 16594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:48.910509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:54093)
0809 04:40:48.910771 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:48.910785 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:48.910887 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:48.911135 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:48.911137 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:48.911318 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:48.911416 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:48.911421 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:48.912125 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:48.912128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:48.912709 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:48.912717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:48.913133 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:48.913136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:48.913230 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:48.913709 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:48.913720 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:48.915779 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":70,"threads_started":1}
W20260809 04:40:48.916077 16592 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:48.910632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:54093 (local address 127.5.222.254:37271)
0809 04:40:48.910794 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:48.910798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:48.910837 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:40:48.910963 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:48.910965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:48.911009 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:48.911068 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:48.911523 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:48.912028 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:48.912824 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:48.913037 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:48.913110 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:48.915944 (+ 2834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:54093: BlockingRecv error: recv got EOF from 127.5.222.195:54093 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 04:40:49 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:49 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:49.515826 16595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:49.510354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:54765)
0809 04:40:49.510628 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:49.510641 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:49.510726 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:49.510969 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:49.510971 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:49.511152 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:49.511247 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:49.511252 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:49.512041 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:49.512044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:49.512621 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:49.512629 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:49.513070 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:49.513073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:49.513131 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:49.513720 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:49.513733 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:49.515628 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":103,"threads_started":1}
W20260809 04:40:49.516008 16596 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:49.510468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:54765 (local address 127.5.222.254:37271)
0809 04:40:49.510736 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:49.510742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:49.510760 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:40:49.510797 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:49.510800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:49.510843 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:49.510936 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:49.511381 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:49.511935 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:49.512730 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:49.512969 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:49.513053 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:49.515829 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:54765: BlockingRecv error: recv got EOF from 127.5.222.193:54765 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":108,"threads_started":1}
Aug 09 04:40:49 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:49 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:49.725437 16598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:49.719681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:56397)
0809 04:40:49.719976 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:49.719989 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:49.720092 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:49.720388 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:49.720390 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:49.720569 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:49.720662 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:49.720667 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:49.721478 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:49.721480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:49.722031 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:49.722039 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:49.722488 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:49.722491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:49.722549 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:49.723114 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:49.723126 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:49.725230 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":96,"threads_started":1}
W20260809 04:40:49.725483 16596 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:49.719759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:56397 (local address 127.5.222.254:37271)
0809 04:40:49.719909 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:49.719915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:49.720064 (+ 149us) server_negotiation.cc:407] Connection header received
0809 04:40:49.720218 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:49.720221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:49.720275 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:49.720355 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:49.720776 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:49.721386 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:49.722139 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:49.722386 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:49.722473 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:49.725376 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:56397: BlockingRecv error: recv got EOF from 127.5.222.194:56397 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 04:40:49 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:49 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:49.926937 16599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:49.916913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:53791)
0809 04:40:49.917282 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:49.917299 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:49.917455 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:49.917732 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:49.917735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:49.917994 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:49.918089 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:49.918095 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:49.918871 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:49.918877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:49.919452 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:49.919459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:49.919917 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:49.919921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:49.920008 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:49.924647 (+ 4639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:49.924660 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:49.926730 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":105,"threads_started":1}
W20260809 04:40:49.927182 16596 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:49.917116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:53791 (local address 127.5.222.254:37271)
0809 04:40:49.917283 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:49.917290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:49.917389 (+ 99us) server_negotiation.cc:407] Connection header received
0809 04:40:49.917559 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:49.917563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:49.917616 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:49.917707 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:49.918216 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:49.918755 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:49.919574 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:49.919801 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:49.919912 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:49.926951 (+ 7039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:53791: BlockingRecv error: recv got EOF from 127.5.222.195:53791 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 04:40:50 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:50 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:50.523943 16600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:50.516804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:38763)
0809 04:40:50.517116 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:50.517132 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:50.517274 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:50.517782 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:50.517785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:50.518061 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:50.518195 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:50.518206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:50.519444 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:50.519447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:50.520015 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:50.520023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:50.520699 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:50.520703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:50.520822 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:50.521506 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:50.521528 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:50.523688 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":111,"threads_started":1}
W20260809 04:40:50.524044 16602 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:50.517256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:38763 (local address 127.5.222.254:37271)
0809 04:40:50.517486 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:50.517493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:50.517510 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:40:50.517549 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:50.517552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:50.517632 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:50.517956 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:50.518367 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:50.519312 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:50.520153 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:50.520524 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:50.520643 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:50.523841 (+ 3198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:38763: BlockingRecv error: recv got EOF from 127.5.222.193:38763 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":72,"threads_started":1}
Aug 09 04:40:50 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:50 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:50.732179 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:50.726411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:53991)
0809 04:40:50.726727 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:50.726744 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:50.726862 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:50.727094 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:50.727097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:50.727283 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:50.727401 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:50.727408 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:50.728257 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:50.728262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:50.728892 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:50.728901 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:50.729403 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:50.729407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:50.729489 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:50.730072 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:50.730094 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:50.732005 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":107,"threads_started":1}
W20260809 04:40:50.732246 16602 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:50.726475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:53991 (local address 127.5.222.254:37271)
0809 04:40:50.726635 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:50.726640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:50.726832 (+ 192us) server_negotiation.cc:407] Connection header received
0809 04:40:50.726947 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:50.726949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:50.726995 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:50.727054 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:50.727506 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:50.728135 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:50.729031 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:50.729283 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:50.729397 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:50.732137 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:53991: BlockingRecv error: recv got EOF from 127.5.222.194:53991 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 04:40:50 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:50 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:50.934108 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:50.927718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:47161)
0809 04:40:50.928049 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:50.928064 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:50.928223 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:50.928561 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:50.928564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:50.928859 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:50.928968 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:50.928976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:50.929905 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:50.929912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:50.930554 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:50.930564 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:50.931120 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:50.931125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:50.931218 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:50.931794 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:50.931808 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:50.933918 (+ 2110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":97,"threads_started":1}
W20260809 04:40:50.934168 16602 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:50.927741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:47161 (local address 127.5.222.254:37271)
0809 04:40:50.927946 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:50.927952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:50.928162 (+ 210us) server_negotiation.cc:407] Connection header received
0809 04:40:50.928354 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:50.928359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:50.928427 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:50.928535 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:50.929116 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:50.929769 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:50.930701 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:50.930983 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:50.931119 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:50.934053 (+ 2934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:47161: BlockingRecv error: recv got EOF from 127.5.222.195:47161 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 04:40:51 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:51 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:51.530704 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:51.525035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:47689)
0809 04:40:51.525392 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:51.525408 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:51.525550 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:51.525843 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:51.525846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:51.526065 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:51.526167 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:51.526173 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:51.526971 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:51.526976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:51.527533 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:51.527541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:51.528015 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:51.528018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:51.528105 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:51.528633 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:51.528644 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:51.530533 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":111,"threads_started":1}
W20260809 04:40:51.530795 16606 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:51.525181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:47689 (local address 127.5.222.254:37271)
0809 04:40:51.525419 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:51.525424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:51.525486 (+ 62us) server_negotiation.cc:407] Connection header received
0809 04:40:51.525653 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:51.525656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:51.525702 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:51.525816 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:51.526296 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:51.526843 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:51.527657 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:51.527890 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:51.528001 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:51.530663 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:47689: BlockingRecv error: recv got EOF from 127.5.222.193:47689 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":64,"threads_started":1}
Aug 09 04:40:51 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:51 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:51.738786 16607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:51.733069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:32951)
0809 04:40:51.733430 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:51.733447 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:51.733602 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:51.733881 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:51.733884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:51.734102 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:51.734195 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:51.734201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:51.734975 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:51.734981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:51.735562 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:51.735570 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:51.736038 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:51.736041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:51.736125 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:51.736621 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:51.736632 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:51.738603 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":114,"threads_started":1}
W20260809 04:40:51.738832 16606 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:51.733215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:32951 (local address 127.5.222.254:37271)
0809 04:40:51.733419 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:51.733424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:51.733539 (+ 115us) server_negotiation.cc:407] Connection header received
0809 04:40:51.733708 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:51.733712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:51.733767 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:51.733856 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:51.734325 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:51.734864 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:51.735690 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:51.735923 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:51.736033 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:51.738728 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:32951: BlockingRecv error: recv got EOF from 127.5.222.194:32951 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 09 04:40:51 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:51 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:51.940683 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:51.934949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:48181)
0809 04:40:51.935238 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:51.935251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:51.935346 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:51.935640 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:51.935642 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:51.935824 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:51.935919 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:51.935924 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:51.936692 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:51.936694 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:51.937267 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:51.937275 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:51.937751 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:51.937755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:51.937840 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:51.938503 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:51.938521 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:51.940558 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":77,"threads_started":1}
W20260809 04:40:51.940855 16606 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:51.935101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:48181 (local address 127.5.222.254:37271)
0809 04:40:51.935290 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:51.935294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:51.935311 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:40:51.935472 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:51.935475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:51.935524 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:51.935600 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:51.936039 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:51.936593 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:51.937396 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:51.937631 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:51.937749 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:51.940695 (+ 2946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:48181: BlockingRecv error: recv got EOF from 127.5.222.195:48181 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 04:40:52 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:52 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:52.537307 16609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:52.531605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:58881)
0809 04:40:52.531996 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:52.532010 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:52.532096 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:52.532386 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:52.532388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:52.532573 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:52.532668 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:52.532673 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:52.533468 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:52.533472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:52.534061 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:52.534070 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:52.534543 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:52.534546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:52.534611 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:52.535213 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:52.535226 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:52.537125 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":170,"threads_started":1}
W20260809 04:40:52.537540 16610 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:52.531735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:58881 (local address 127.5.222.254:37271)
0809 04:40:52.532031 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:52.532035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:52.532052 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:40:52.532203 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:52.532205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:52.532267 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:52.532346 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:52.532786 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:52.533359 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:52.534177 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:52.534413 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:52.534491 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:52.537393 (+ 2902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:58881: BlockingRecv error: recv got EOF from 127.5.222.193:58881 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":92,"threads_started":1}
Aug 09 04:40:52 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:52 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:52.745476 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:52.739613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:45789)
0809 04:40:52.739929 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:52.739947 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:52.740056 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:52.740343 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:52.740346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:52.740566 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:52.740655 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:52.740661 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:52.741534 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:52.741539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:52.742115 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:52.742124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:52.742593 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:52.742596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:52.742683 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:52.743211 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:52.743225 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:52.745279 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":111,"threads_started":1}
W20260809 04:40:52.745563 16610 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:52.739777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:45789 (local address 127.5.222.254:37271)
0809 04:40:52.739933 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:52.739939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:52.740005 (+ 66us) server_negotiation.cc:407] Connection header received
0809 04:40:52.740165 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:52.740168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:52.740223 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:52.740314 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:52.740835 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:52.741425 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:52.742244 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:52.742471 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:52.742579 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:52.745432 (+ 2853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:45789: BlockingRecv error: recv got EOF from 127.5.222.194:45789 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 04:40:52 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:52 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:52.947197 16612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:52.941540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:47099)
0809 04:40:52.941829 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:52.941843 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:52.941944 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:52.942185 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:52.942187 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:52.942377 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:52.942472 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:52.942477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:52.943303 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:52.943306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:52.943870 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:52.943878 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:52.944348 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:52.944350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:52.944410 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:52.944936 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:52.944947 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:52.947002 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":115,"threads_started":1}
W20260809 04:40:52.947247 16610 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:52.941675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:47099 (local address 127.5.222.254:37271)
0809 04:40:52.941861 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:52.941865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:52.941894 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:40:52.942029 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:52.942032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:52.942076 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:52.942148 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:52.942585 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:52.943202 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:52.943995 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:52.944243 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:52.944331 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:52.947142 (+ 2811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:47099: BlockingRecv error: recv got EOF from 127.5.222.195:47099 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 09 04:40:53 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:53 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:53.544013 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:53.538173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:56523)
0809 04:40:53.538504 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:53.538522 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:53.538674 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:53.538984 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:53.538988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:53.539206 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:53.539304 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:53.539310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:53.540114 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:53.540119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:53.540690 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:53.540699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:53.541207 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:53.541211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:53.541300 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:53.541904 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:53.541923 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:53.543893 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":110,"threads_started":1}
W20260809 04:40:53.544229 16613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:53.538204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:56523 (local address 127.5.222.254:37271)
0809 04:40:53.538505 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:53.538510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:53.538612 (+ 102us) server_negotiation.cc:407] Connection header received
0809 04:40:53.538778 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:53.538782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:53.538838 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:53.538931 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:53.539440 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:53.539997 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:53.540816 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:53.541058 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:53.541194 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:53.544042 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:56523: BlockingRecv error: recv got EOF from 127.5.222.193:56523 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":91,"threads_started":1}
Aug 09 04:40:53 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:53 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:53.752516 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:53.746456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:44771)
0809 04:40:53.746749 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:53.746767 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:53.746866 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:53.747146 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:53.747148 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:53.747363 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:53.747459 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:53.747464 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:53.748321 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:53.748324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:53.749031 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:53.749042 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:53.749592 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:53.749595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:53.749765 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:53.750384 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:53.750397 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:53.752396 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":83,"threads_started":1}
W20260809 04:40:53.752702 16613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:53.746595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:44771 (local address 127.5.222.254:37271)
0809 04:40:53.746793 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:53.746798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:53.746816 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:40:53.746962 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:53.746965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:53.747013 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:53.747105 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:53.747599 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:53.748175 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:53.749189 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:53.749455 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:53.749560 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:53.752562 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:44771: BlockingRecv error: recv got EOF from 127.5.222.194:44771 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 04:40:53 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:53 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:53.954713 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:53.948071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:38145)
0809 04:40:53.948526 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:53.948539 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:53.948652 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:53.949013 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:53.949015 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:53.949252 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:53.949371 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:53.949377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:53.950384 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:53.950390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:53.951203 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:53.951214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:53.951838 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:53.951841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:53.951918 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:53.952444 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:53.952457 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:53.954533 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":150,"threads_started":1}
W20260809 04:40:53.954777 16613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:53.948205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:38145 (local address 127.5.222.254:37271)
0809 04:40:53.948372 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:53.948376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:53.948628 (+ 252us) server_negotiation.cc:407] Connection header received
0809 04:40:53.948822 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:53.948825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:53.948882 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:53.948975 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:53.949558 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:53.950251 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:53.951343 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:53.951605 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:53.951763 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:53.954662 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:38145: BlockingRecv error: recv got EOF from 127.5.222.195:38145 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 04:40:54 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:54 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:54.551047 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:54.544960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:33075)
0809 04:40:54.545424 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:54.545438 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:54.545543 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:54.545785 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:54.545788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:54.545977 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:54.546072 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:54.546077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:54.546894 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:54.546897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:54.547472 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:54.547480 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:54.548032 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:54.548036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:54.548130 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:54.548747 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:54.548760 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:54.550888 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":199,"threads_started":1}
W20260809 04:40:54.551184 16619 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:54.545065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:33075 (local address 127.5.222.254:37271)
0809 04:40:54.545410 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:54.545415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:54.545545 (+ 130us) server_negotiation.cc:407] Connection header received
0809 04:40:54.545617 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:54.545620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:54.545665 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:54.545743 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:54.546176 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:54.546792 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:54.547581 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:54.547895 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:54.548027 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:54.551045 (+ 3018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:33075: BlockingRecv error: recv got EOF from 127.5.222.193:33075 (error 108)
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":138,"threads_started":1}
Aug 09 04:40:54 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:54 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:54.758711 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:54.753333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:53213)
0809 04:40:54.753688 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:54.753702 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:54.753807 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:54.754031 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:54.754033 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:54.754225 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:54.754336 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:54.754342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:54.755100 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:54.755103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:54.755668 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:54.755676 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:54.756121 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:54.756124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:54.756187 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:54.756692 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:54.756703 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:54.758540 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":122,"threads_started":1}
W20260809 04:40:54.758828 16619 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:54.753388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:53213 (local address 127.5.222.254:37271)
0809 04:40:54.753555 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:54.753560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:54.753779 (+ 219us) server_negotiation.cc:407] Connection header received
0809 04:40:54.753877 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:54.753879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:54.753924 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:54.753993 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:54.754441 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:54.754994 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:54.755794 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:54.756010 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:54.756085 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:54.758665 (+ 2580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:53213: BlockingRecv error: recv got EOF from 127.5.222.194:53213 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 04:40:54 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:54 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:54.967337 16621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:54.955671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:60229)
0809 04:40:54.956130 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:54.956144 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:54.956294 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:54.956620 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:54.956623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:54.956862 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:54.956962 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:54.956968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:54.957853 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:54.957857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:54.958421 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:54.958431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:54.958930 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:54.958935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:54.959028 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:54.964582 (+ 5554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:54.964596 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:54.967184 (+ 2588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":133,"threads_started":1}
W20260809 04:40:54.967579 16619 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:54.955742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:60229 (local address 127.5.222.254:37271)
0809 04:40:54.955928 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:54.955934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:54.956241 (+ 307us) server_negotiation.cc:407] Connection header received
0809 04:40:54.956431 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:54.956434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:54.956495 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:54.956591 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:54.957105 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:54.957716 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:54.958553 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:54.958801 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:54.958926 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:54.967404 (+ 8478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:60229: BlockingRecv error: recv got EOF from 127.5.222.195:60229 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 04:40:55 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:55 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:55.559031 16622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:55.551983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:51871)
0809 04:40:55.553388 (+ 1405us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:55.553404 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:55.553535 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:55.553819 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:55.553822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:55.554044 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:55.554137 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:55.554143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:55.554938 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:55.554941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:55.555505 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:55.555513 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:55.556004 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:55.556007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:55.556093 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:55.556681 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:55.556692 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:55.558856 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1299,"thread_start_us":93,"threads_started":1}
W20260809 04:40:55.559127 16623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:55.552173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:51871 (local address 127.5.222.254:37271)
0809 04:40:55.552514 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:55.552519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:55.553495 (+ 976us) server_negotiation.cc:407] Connection header received
0809 04:40:55.553637 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:55.553640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:55.553697 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:55.553794 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:55.554259 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:55.554815 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:55.555635 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:55.555880 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:55.555986 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:55.558990 (+ 3004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:51871: BlockingRecv error: recv got EOF from 127.5.222.193:51871 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":80,"threads_started":1}
Aug 09 04:40:55 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:55 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:55.766130 16624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:55.759630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:41557)
0809 04:40:55.760021 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:55.760049 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:55.760207 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:55.760532 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:55.760536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:55.760773 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:55.760872 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:55.760879 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:55.761992 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:55.761999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:55.762806 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:55.762816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:55.763311 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:55.763315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:55.763401 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:55.763919 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:55.763932 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:55.765934 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":91,"threads_started":1}
W20260809 04:40:55.766368 16623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:55.759761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:41557 (local address 127.5.222.254:37271)
0809 04:40:55.759975 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:55.759980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:55.760147 (+ 167us) server_negotiation.cc:407] Connection header received
0809 04:40:55.760329 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:55.760334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:55.760393 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:55.760495 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:55.761008 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:55.761863 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:55.762939 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:55.763179 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:55.763310 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:55.766153 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:41557: BlockingRecv error: recv got EOF from 127.5.222.194:41557 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 09 04:40:55 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:55 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:55.973950 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:55.968214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:39817)
0809 04:40:55.968527 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:55.968540 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:55.968638 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:55.968881 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:55.968883 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:55.969064 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:55.969199 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:55.969205 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:55.969962 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:55.969965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:55.970521 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:55.970529 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:55.970967 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:55.970970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:55.971031 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:55.971549 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:55.971562 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:55.973834 (+ 2272us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":128,"threads_started":1}
W20260809 04:40:55.974133 16623 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:55.968322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:39817 (local address 127.5.222.254:37271)
0809 04:40:55.968457 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:55.968461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:55.968610 (+ 149us) server_negotiation.cc:407] Connection header received
0809 04:40:55.968720 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:55.968722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:55.968771 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:55.968842 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:55.969324 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:55.969861 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:55.970634 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:55.970864 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:55.970948 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:55.973991 (+ 3043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:39817: BlockingRecv error: recv got EOF from 127.5.222.195:39817 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 04:40:56 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:56 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:56.565595 16627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:56.559883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:59429)
0809 04:40:56.560174 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:56.560192 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:56.560297 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:56.560533 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:56.560535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:56.560715 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:56.560820 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:56.560825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:56.561807 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:56.561811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:56.562374 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:56.562382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:56.562869 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:56.562872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:56.562935 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:56.563525 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:56.563537 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:56.565474 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
W20260809 04:40:56.565748 16628 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:56.560026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:59429 (local address 127.5.222.254:37271)
0809 04:40:56.560281 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:56.560285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:56.560298 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:40:56.560378 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:56.560380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:56.560425 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:56.560494 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:56.560961 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:56.561617 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:56.562490 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:56.562760 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:56.562854 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:56.565610 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:59429: BlockingRecv error: recv got EOF from 127.5.222.193:59429 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":85,"threads_started":1}
Aug 09 04:40:56 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:56 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:56.773531 16629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:56.767022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:42743)
0809 04:40:56.767393 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:56.767409 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:56.767526 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:56.767856 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:56.767859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:56.768072 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:56.768178 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:56.768184 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:56.769242 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:56.769248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:56.770119 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:56.770129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:56.770563 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:56.770566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:56.770631 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:56.771309 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:56.771322 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:56.773368 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":113,"threads_started":1}
W20260809 04:40:56.773610 16628 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:56.767174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:42743 (local address 127.5.222.254:37271)
0809 04:40:56.767343 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:56.767346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:56.767493 (+ 147us) server_negotiation.cc:407] Connection header received
0809 04:40:56.767612 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:56.767615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:56.767662 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:56.767733 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:56.768319 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:56.769057 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:56.770233 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:56.770454 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:56.770536 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:56.773514 (+ 2978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:42743: BlockingRecv error: recv got EOF from 127.5.222.194:42743 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 04:40:56 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:56 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:56.980475 16630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:56.974837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:42941)
0809 04:40:56.975156 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:56.975168 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:56.975278 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:56.975576 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:56.975578 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:56.975785 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:56.975888 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:56.975894 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:56.976656 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:56.976658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:56.977295 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:56.977304 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:56.977721 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:56.977724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:56.977821 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:56.978418 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:56.978430 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:56.980335 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":89,"threads_started":1}
W20260809 04:40:56.980618 16628 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:56.974891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:42941 (local address 127.5.222.254:37271)
0809 04:40:56.975030 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:56.975034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:56.975238 (+ 204us) server_negotiation.cc:407] Connection header received
0809 04:40:56.975357 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:56.975359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:56.975417 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:56.975488 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:56.976004 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:56.976553 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:56.977397 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:56.977623 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:56.977701 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:56.980487 (+ 2786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:42941: BlockingRecv error: recv got EOF from 127.5.222.195:42941 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 04:40:57 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:57 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:57.572373 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:57.566536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:55927)
0809 04:40:57.566900 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:57.566914 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:57.567006 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:57.567377 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:57.567380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:57.567604 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:57.567704 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:57.567710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:57.568562 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:57.568567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:57.569212 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:57.569226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:57.569692 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:57.569695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:57.569790 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:57.570364 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:57.570376 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:57.572247 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":138,"threads_started":1}
W20260809 04:40:57.572624 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:57.566659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:55927 (local address 127.5.222.254:37271)
0809 04:40:57.566942 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:57.566946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:57.566965 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:40:57.567149 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:57.567152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:57.567208 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:57.567315 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:57.567872 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:57.568443 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:57.569346 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:57.569576 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:57.569688 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:57.572460 (+ 2772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:55927: BlockingRecv error: recv got EOF from 127.5.222.193:55927 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":99,"threads_started":1}
Aug 09 04:40:57 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:57 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:57.780156 16634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:57.774434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:33793)
0809 04:40:57.774718 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:57.774732 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:57.774826 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:57.775072 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:57.775074 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:57.775256 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:57.775352 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:57.775358 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:57.776155 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:57.776157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:57.776714 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:57.776722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:57.777289 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:57.777292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:57.777359 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:57.777891 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:57.777903 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:57.779979 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":101,"threads_started":1}
W20260809 04:40:57.780216 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:57.774552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:33793 (local address 127.5.222.254:37271)
0809 04:40:57.774700 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:57.774704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:57.774803 (+ 99us) server_negotiation.cc:407] Connection header received
0809 04:40:57.774912 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:57.774914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:57.774959 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:57.775029 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:57.775494 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:57.776043 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:57.776870 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:57.777097 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:57.777276 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:57.780106 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:33793: BlockingRecv error: recv got EOF from 127.5.222.194:33793 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 04:40:57 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:57 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:57.987080 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:57.981365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:52245)
0809 04:40:57.981640 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:57.981654 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:57.981781 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:57.982063 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:57.982066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:57.982270 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:57.982367 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:57.982373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:57.983158 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:57.983163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:57.983753 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:57.983762 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:57.984240 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:57.984244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:57.984330 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:57.984929 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:57.984940 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:57.986908 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":98,"threads_started":1}
W20260809 04:40:57.987171 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:57.981621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:52245 (local address 127.5.222.254:37271)
0809 04:40:57.981784 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:57.981789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:57.981805 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:40:57.981875 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:57.981878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:57.981939 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:57.982029 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:57.982505 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:57.983042 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:57.983883 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:57.984116 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:57.984226 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:57.987048 (+ 2822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:52245: BlockingRecv error: recv got EOF from 127.5.222.195:52245 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 04:40:58 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:58 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:58.578636 16636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:58.573190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:49967)
0809 04:40:58.573499 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:58.573514 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:58.573607 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:58.573848 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:58.573851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:58.574039 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:58.574140 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:58.574145 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:58.574939 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:58.574942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:58.575497 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:58.575504 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:58.575980 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:58.575983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:58.576045 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:58.576538 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:58.576549 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:58.578493 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":115,"threads_started":1}
W20260809 04:40:58.579183 16637 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:58.573351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:49967 (local address 127.5.222.254:37271)
0809 04:40:58.573577 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:58.573580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:58.573594 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:40:58.573678 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:58.573680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:58.573722 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:58.573809 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:58.574261 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:58.574830 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:58.575613 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:58.575867 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:58.575951 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:58.578655 (+ 2704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:49967: BlockingRecv error: recv got EOF from 127.5.222.193:49967 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":62,"threads_started":1}
Aug 09 04:40:58 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:58 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:58.786284 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:58.780957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:60295)
0809 04:40:58.781245 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:58.781261 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:58.781372 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:58.781618 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:58.781621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:58.781805 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:58.781903 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:58.781908 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:58.782636 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:58.782638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:58.783187 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:58.783195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:58.783625 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:58.783627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:58.783689 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:58.784172 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:58.784183 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:58.786170 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":95,"threads_started":1}
W20260809 04:40:58.786481 16637 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:58.781018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:60295 (local address 127.5.222.254:37271)
0809 04:40:58.781240 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:58.781245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:58.781377 (+ 132us) server_negotiation.cc:407] Connection header received
0809 04:40:58.781453 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:58.781456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:58.781504 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:58.781579 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:58.782014 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:58.782540 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:58.783300 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:58.783524 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:58.783602 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:58.786328 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:60295: BlockingRecv error: recv got EOF from 127.5.222.194:60295 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 04:40:58 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:58 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:58.994385 16641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:58.988147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:52037)
0809 04:40:58.988498 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:58.988516 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:58.988658 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:58.988990 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:58.988994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:58.989337 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:58.989477 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:58.989484 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:58.990601 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:58.990607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:58.991243 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:58.991253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:58.991742 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:58.991746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:58.991839 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:58.992334 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:58.992347 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:58.994261 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":101,"threads_started":1}
W20260809 04:40:58.994516 16637 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:58.988289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:52037 (local address 127.5.222.254:37271)
0809 04:40:58.988433 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:58.988439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:58.988612 (+ 173us) server_negotiation.cc:407] Connection header received
0809 04:40:58.988800 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:58.988803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:58.988864 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:58.988962 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:58.989608 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:58.990307 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:58.991374 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:58.991612 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:58.991731 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:58.994382 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:52037: BlockingRecv error: recv got EOF from 127.5.222.195:52037 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 04:40:59 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:59 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:59.584985 16642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:59.579424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:39417)
0809 04:40:59.579716 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:59.579730 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:59.579822 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:59.580071 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:59.580073 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:59.580256 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:59.580355 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:59.580360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:59.581248 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:59.581250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:59.581799 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:59.581806 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:59.582278 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:59.582280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:59.582341 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:59.582895 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:59.582918 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:59.584861 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":109,"threads_started":1}
W20260809 04:40:59.585270 16643 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:59.579538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:39417 (local address 127.5.222.254:37271)
0809 04:40:59.579840 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:59.579844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:59.579858 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:40:59.579897 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:59.579899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:59.579943 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:59.580038 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:59.580481 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:59.581117 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:59.581938 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:59.582169 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:59.582259 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:59.585058 (+ 2799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:39417: BlockingRecv error: recv got EOF from 127.5.222.193:39417 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":93,"threads_started":1}
Aug 09 04:40:59 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:40:59 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:40:59.792781 16644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:59.787124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:53691)
0809 04:40:59.787559 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:59.787572 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:59.787669 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:59.787910 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:59.787913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:59.788098 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:59.788194 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:59.788200 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:59.788917 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:59.788919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:59.789555 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:59.789565 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:59.789965 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:59.789968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:59.790026 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:59.790607 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:40:59.790619 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:40:59.792655 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":85,"threads_started":1}
W20260809 04:40:59.793020 16643 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:59.787195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:53691 (local address 127.5.222.254:37271)
0809 04:40:59.787406 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:59.787411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:59.787649 (+ 238us) server_negotiation.cc:407] Connection header received
0809 04:40:59.787749 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:59.787751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:59.787799 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:59.787868 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:59.788296 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:59.788821 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:59.789664 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:59.789870 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:59.789947 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:40:59.792795 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:53691: BlockingRecv error: recv got EOF from 127.5.222.194:53691 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 09 04:41:00 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:41:00 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:41:00.010020 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:40:59.995518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:35099)
0809 04:40:59.995870 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 04:40:59.995884 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:40:59.996031 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:40:59.996419 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:40:59.996425 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:40:59.996672 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:40:59.996807 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:59.996816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:59.997823 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:59.997827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:59.998413 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:40:59.998422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:40:59.998896 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:40:59.998898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:40:59.998964 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:00.007144 (+ 8180us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:00.007163 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:00.009893 (+ 2730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":127,"threads_started":1}
W20260809 04:41:00.010236 16643 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:40:59.995585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:35099 (local address 127.5.222.254:37271)
0809 04:40:59.995762 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:40:59.995767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:40:59.995981 (+ 214us) server_negotiation.cc:407] Connection header received
0809 04:40:59.996152 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:40:59.996157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:40:59.996255 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:40:59.996360 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:40:59.996953 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:59.997707 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:59.998537 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:40:59.998784 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:40:59.998878 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:00.010092 (+ 11214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:35099: BlockingRecv error: recv got EOF from 127.5.222.195:35099 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 04:41:00 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:41:00 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:41:00.592756 16646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:00.585982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:54437)
0809 04:41:00.586298 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:00.586312 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:00.586412 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:00.586728 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:00.586731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:00.586993 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:00.587131 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:00.587139 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:00.588274 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:00.588280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:00.589373 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:00.589389 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:00.589865 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:00.589871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:00.589991 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:00.590539 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:00.590560 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:00.592640 (+ 2080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":98,"threads_started":1}
W20260809 04:41:00.593228 16647 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:41:00.586052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:54437 (local address 127.5.222.254:37271)
0809 04:41:00.586350 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:00.586354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:00.586370 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:41:00.586542 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:00.586545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:00.586604 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:00.586705 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:00.587260 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:00.588034 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:00.589510 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:00.589736 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:00.589819 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:00.592980 (+ 3161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:54437: BlockingRecv error: recv got EOF from 127.5.222.193:54437 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":105,"threads_started":1}
Aug 09 04:41:00 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:41:00 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:41:00.799564 16648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:00.793678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:51711)
0809 04:41:00.794024 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:00.794042 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:00.794164 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:00.794463 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:00.794466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:00.794689 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:00.794784 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:00.794790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:00.795583 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:00.795588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:00.796170 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:00.796179 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:00.796650 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:00.796654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:00.796734 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:00.797336 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:00.797353 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:00.799373 (+ 2020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":113,"threads_started":1}
W20260809 04:41:00.799600 16647 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:41:00.793843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:51711 (local address 127.5.222.254:37271)
0809 04:41:00.794028 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:00.794035 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:00.794103 (+ 68us) server_negotiation.cc:407] Connection header received
0809 04:41:00.794279 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:00.794283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:00.794338 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:00.794428 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:00.794921 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:00.795467 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:00.796309 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:00.796534 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:00.796646 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:00.799493 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:51711: BlockingRecv error: recv got EOF from 127.5.222.194:51711 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 09 04:41:01 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:41:01 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:41:01.016664 16649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:01.011058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:36017)
0809 04:41:01.011358 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:01.011376 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:01.011496 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:01.011838 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:01.011840 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:01.012117 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:01.012271 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:01.012284 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:01.013040 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:01.013043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:01.013653 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:01.013661 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:01.014089 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:01.014091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:01.014156 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:01.014633 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:01.014645 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:01.016544 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":92,"threads_started":1}
W20260809 04:41:01.016804 16647 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:41:01.011126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:36017 (local address 127.5.222.254:37271)
0809 04:41:01.011284 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:01.011291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:01.011477 (+ 186us) server_negotiation.cc:407] Connection header received
0809 04:41:01.011586 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:01.011590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:01.011656 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:01.011725 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:01.012393 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:01.012936 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:01.013760 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:01.013986 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:01.014065 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:01.016675 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:36017: BlockingRecv error: recv got EOF from 127.5.222.195:36017 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 04:41:01 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:41:01 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:41:01.599833 16651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:01.593681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:51067)
0809 04:41:01.594019 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:01.594034 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:01.594136 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:01.594406 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:01.594408 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:01.594605 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:01.594706 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:01.594711 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:01.595582 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:01.595585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:01.596247 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:01.596256 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:01.596751 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:01.596754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:01.596817 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:01.597516 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:01.597529 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:01.599683 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":105,"threads_started":1}
W20260809 04:41:01.599951 16652 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:41:01.593844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:51067 (local address 127.5.222.254:37271)
0809 04:41:01.594183 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:01.594187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:01.594201 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:41:01.594239 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:01.594241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:01.594282 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:01.594362 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:01.594873 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:01.595468 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:01.596381 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:01.596638 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:01.596731 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:01.599820 (+ 3089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:51067: BlockingRecv error: recv got EOF from 127.5.222.193:51067 (error 108)
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":97,"threads_started":1}
Aug 09 04:41:01 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:41:01 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:41:01.806500 16654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:01.800662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:52911)
0809 04:41:01.800968 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:01.800981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:01.801087 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:01.801384 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:01.801386 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:01.801574 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:01.801670 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:01.801675 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:01.802504 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:01.802508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:01.803161 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:01.803171 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:01.803611 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:01.803615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:01.803732 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:01.804252 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:01.804266 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:01.806288 (+ 2022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":108,"threads_started":1}
W20260809 04:41:01.806540 16652 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:41:01.800706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:52911 (local address 127.5.222.254:37271)
0809 04:41:01.800869 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:01.800874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:01.801061 (+ 187us) server_negotiation.cc:407] Connection header received
0809 04:41:01.801203 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:01.801209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:01.801263 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:01.801333 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:01.801775 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:01.802299 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:01.803286 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:01.803499 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:01.803574 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:01.806417 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:52911: BlockingRecv error: recv got EOF from 127.5.222.194:52911 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 04:41:02 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:41:02 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.195@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:41:02.023151 16655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:02.017609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.195:57271)
0809 04:41:02.017884 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:02.017898 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:02.017986 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:02.018241 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:02.018243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:02.018443 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:02.018538 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:02.018543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:02.019393 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:02.019395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:02.019960 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:02.019967 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:02.020514 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:02.020517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:02.020578 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:02.021141 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:02.021206 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:02.023023 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":103,"threads_started":1}
W20260809 04:41:02.023398 16652 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:41:02.017802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:57271 (local address 127.5.222.254:37271)
0809 04:41:02.017967 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:02.017972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:02.017989 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:41:02.018060 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:02.018062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:02.018120 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:02.018227 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:02.018682 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:02.019287 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:02.020075 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:02.020400 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:02.020505 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:02.023229 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.195:57271: BlockingRecv error: recv got EOF from 127.5.222.195:57271 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 04:41:02 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:41:02 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.193@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:41:02.606606 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:02.600782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.193:33139)
0809 04:41:02.601120 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:02.601136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:02.601335 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:02.601651 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:02.601653 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:02.601867 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:02.601975 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:02.601981 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:02.602746 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:02.602749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:02.603345 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:02.603353 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:02.603810 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:02.603813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:02.603875 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:02.604470 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:02.604483 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:02.606479 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":120,"threads_started":1}
W20260809 04:41:02.606782 16657 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:41:02.600982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:33139 (local address 127.5.222.254:37271)
0809 04:41:02.601267 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:02.601271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:02.601284 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:41:02.601464 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:02.601472 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:02.601519 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:02.601597 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:02.602093 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:02.602625 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:02.603453 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:02.603677 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:02.603751 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:02.606609 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.193:33139: BlockingRecv error: recv got EOF from 127.5.222.193:33139 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":109,"threads_started":1}
Aug 09 04:41:02 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:41:02 dist-test-slave-vxj2 krb5kdc[15434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.222.194@KRBTEST.COM for oryx/127.5.222.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:41:02.814340 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:02.807513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:37271 (local address 127.5.222.194:46419)
0809 04:41:02.807912 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:02.807929 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:02.808074 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:02.808412 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:02.808416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:02.808677 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:02.808787 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:02.808794 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:02.809756 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:02.809761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:02.810424 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:02.810434 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:02.811269 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:02.811273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:02.811403 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:02.812046 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:02.812059 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:02.814122 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.222.254:37271: Server oryx/127.5.222.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":89,"threads_started":1}
W20260809 04:41:02.814378 16657 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:41:02.807662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:46419 (local address 127.5.222.254:37271)
0809 04:41:02.807840 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:02.807846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:02.808023 (+ 177us) server_negotiation.cc:407] Connection header received
0809 04:41:02.808192 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:02.808197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:02.808279 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:02.808381 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:02.808931 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:02.809622 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:02.810568 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:02.810864 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:02.811276 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:02.814263 (+ 2987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.222.194:46419: BlockingRecv error: recv got EOF from 127.5.222.194:46419 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:41:02.897796 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 16105
I20260809 04:41:02.905189 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 16285
I20260809 04:41:02.913053 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 16439
I20260809 04:41:02.920171 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 16024
2026-08-09T04:41:02Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18119 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T04:41:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:41:02Z Disabled control of system clock
I20260809 04:41:02.953195 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:32975
--webserver_interface=127.5.222.254
--webserver_port=0
--builtin_ntp_servers=127.5.222.212:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:32975
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 04:41:03.065516 16662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:03.065784 16662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:03.065819 16662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:03.069851 16662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:41:03.069917 16662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:03.069933 16662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:03.069952 16662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:41:03.069967 16662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:41:03.074676 16662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:32975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:32975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:03.075753 16662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:03.077077 16662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:03.086370 16667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:03.086215 16669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:03.086290 16671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:03.086237 16662 server_base.cc:1034] running on GCE node
I20260809 04:41:03.087167 16662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:03.088415 16662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:03.089622 16662 hybrid_clock.cc:648] HybridClock initialized: now 1786250463089603 us; error 34 us; skew 500 ppm
I20260809 04:41:03.091605 16662 webserver.cc:460] Webserver started at http://127.5.222.254:37209/ using document root <none> and password file <none>
I20260809 04:41:03.092253 16662 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:03.092362 16662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:03.092620 16662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:41:03.095115 16662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "c3d518272e404352a07d1edcfdbc98a8"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "72435e791f499f74b7096f3eee8086b5"
server_key_iv: "821f9a22b46acdd303adf38af4d3005e"
server_key_version: "encryptionkey@0"
I20260809 04:41:03.095755 16662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "c3d518272e404352a07d1edcfdbc98a8"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "72435e791f499f74b7096f3eee8086b5"
server_key_iv: "821f9a22b46acdd303adf38af4d3005e"
server_key_version: "encryptionkey@0"
I20260809 04:41:03.099763 16662 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:41:03.102355 16678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:03.103665 16662 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:41:03.103839 16662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "c3d518272e404352a07d1edcfdbc98a8"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "72435e791f499f74b7096f3eee8086b5"
server_key_iv: "821f9a22b46acdd303adf38af4d3005e"
server_key_version: "encryptionkey@0"
I20260809 04:41:03.103979 16662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:03.122786 16662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:03.123605 16662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:03.123862 16662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:03.132193 16662 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:32975
I20260809 04:41:03.132208 16740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:32975 every 8 connection(s)
I20260809 04:41:03.133487 16662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:41:03.136366 16741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:41:03.139438 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 16662
I20260809 04:41:03.139600 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:41:03.139986 6011 external_mini_cluster.cc:1428] Setting key 586974533563b55e9d234514c4aaac9f
I20260809 04:41:03.143110 16741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8: Bootstrap starting.
I20260809 04:41:03.145917 16741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8: Neither blocks nor log segments found. Creating new log.
I20260809 04:41:03.146746 16741 log.cc:824] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:41:03.149113 16741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8: No bootstrap required, opened a new log
I20260809 04:41:03.150879 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:03.141664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47960 (local address 127.5.222.254:32975)
0809 04:41:03.142401 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:03.142435 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:03.142478 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:41:03.143281 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:03.143303 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:03.143794 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:03.144177 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:03.144669 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.146122 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.147181 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.147758 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.147910 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:03.148780 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:03.148787 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:03.148802 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:41:03.149330 (+ 528us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:03.149397 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:41:03.149438 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:03.149540 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:03.149697 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 04:41:03.149954 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":120,"threads_started":1}
I20260809 04:41:03.153496 16741 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3d518272e404352a07d1edcfdbc98a8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 32975 } }
I20260809 04:41:03.153704 16741 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:41:03.153738 16741 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d518272e404352a07d1edcfdbc98a8, State: Initialized, Role: FOLLOWER
I20260809 04:41:03.154089 16741 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3d518272e404352a07d1edcfdbc98a8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 32975 } }
I20260809 04:41:03.154196 16741 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:41:03.154241 16741 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:41:03.154309 16741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:41:03.155391 16741 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3d518272e404352a07d1edcfdbc98a8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 32975 } }
I20260809 04:41:03.155731 16741 leader_election.cc:302] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3d518272e404352a07d1edcfdbc98a8; no voters:
I20260809 04:41:03.156034 16741 leader_election.cc:288] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:41:03.156167 16746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:41:03.156414 16746 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [term 1 LEADER]: Becoming Leader. State: Replica: c3d518272e404352a07d1edcfdbc98a8, State: Running, Role: LEADER
I20260809 04:41:03.156782 16746 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3d518272e404352a07d1edcfdbc98a8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 32975 } }
I20260809 04:41:03.157368 16741 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:41:03.158650 16747 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3d518272e404352a07d1edcfdbc98a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3d518272e404352a07d1edcfdbc98a8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 32975 } } }
I20260809 04:41:03.158792 16747 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:41:03.159085 16748 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3d518272e404352a07d1edcfdbc98a8. Latest consensus state: current_term: 1 leader_uuid: "c3d518272e404352a07d1edcfdbc98a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3d518272e404352a07d1edcfdbc98a8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 32975 } } }
I20260809 04:41:03.159189 16748 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:41:03.159991 16758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:41:03.162955 16758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:41:03.168498 16758 catalog_manager.cc:1347] Generated new cluster ID: ea93839ce9bc4794ae38b27cf66c0cb7
I20260809 04:41:03.168568 16758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:41:03.190245 16758 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:41:03.191586 16758 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:41:03.195437 16758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3d518272e404352a07d1edcfdbc98a8: Generated new TSK 0
I20260809 04:41:03.196198 16758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:41:03.204802 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:32975
--builtin_ntp_servers=127.5.222.212:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 04:41:03.319537 16767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:03.319885 16767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:03.319972 16767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:03.324671 16767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:03.324774 16767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:03.324877 16767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:41:03.330367 16767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:32975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:03.331606 16767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:03.333235 16767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:03.343616 16774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:03.343616 16776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:03.343700 16767 server_base.cc:1034] running on GCE node
W20260809 04:41:03.343645 16773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:03.344259 16767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:03.344938 16767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:03.346124 16767 hybrid_clock.cc:648] HybridClock initialized: now 1786250463346101 us; error 49 us; skew 500 ppm
I20260809 04:41:03.347980 16767 webserver.cc:460] Webserver started at http://127.5.222.193:41001/ using document root <none> and password file <none>
I20260809 04:41:03.348567 16767 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:03.348662 16767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:03.348902 16767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:41:03.351022 16767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "72202a8707a447f0bb28ccacfcf4ffa3"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "dc1a1c8cb443499efdcbb8e29888ed9c"
server_key_iv: "3932043311a8c91d4e640d5af8627e36"
server_key_version: "encryptionkey@0"
I20260809 04:41:03.351598 16767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "72202a8707a447f0bb28ccacfcf4ffa3"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "dc1a1c8cb443499efdcbb8e29888ed9c"
server_key_iv: "3932043311a8c91d4e640d5af8627e36"
server_key_version: "encryptionkey@0"
I20260809 04:41:03.355213 16767 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:41:03.357733 16782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:03.359103 16767 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:41:03.359237 16767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "72202a8707a447f0bb28ccacfcf4ffa3"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "dc1a1c8cb443499efdcbb8e29888ed9c"
server_key_iv: "3932043311a8c91d4e640d5af8627e36"
server_key_version: "encryptionkey@0"
I20260809 04:41:03.359352 16767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:03.375025 16767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:03.375837 16767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:03.376057 16767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:03.376645 16767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:41:03.377632 16767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:41:03.377681 16767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:03.377720 16767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:41:03.377736 16767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:03.389940 16767 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:42789
I20260809 04:41:03.391130 16767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:41:03.393498 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 16767
I20260809 04:41:03.393613 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:41:03.393957 6011 external_mini_cluster.cc:1428] Setting key f63036a69e6963b4d7e192c8b2a2c7b6
I20260809 04:41:03.394476 16915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:42789 every 8 connection(s)
I20260809 04:41:03.396679 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:32975
--builtin_ntp_servers=127.5.222.212:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 04:41:03.405642 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:03.396455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:48709 (local address 127.5.222.254:32975)
0809 04:41:03.396578 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:03.396582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:03.398903 (+ 2321us) server_negotiation.cc:407] Connection header received
0809 04:41:03.400783 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:03.400790 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:03.400861 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:03.400972 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:03.401938 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.402473 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.403633 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.403878 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.403993 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:03.405114 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:03.405117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:03.405120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:41:03.405134 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:03.405186 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:41:03.405199 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:03.405267 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:03.405432 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 04:41:03.405496 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:41:03.406494 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:03.396771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:32975 (local address 127.5.222.193:48709)
0809 04:41:03.398712 (+ 1941us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:03.398773 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:03.400556 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:03.401125 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:03.401131 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:03.401279 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:03.401776 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:03.401789 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.402607 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.402610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:03.403494 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:03.403504 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.404032 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.404036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:03.404216 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:03.404844 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:41:03.404846 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:03.404884 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:41:03.404953 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:41:03.405268 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:41:03.405279 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:41:03.405287 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:41:03.405425 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 04:41:03.405699 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1409,"thread_start_us":165,"threads_started":1}
I20260809 04:41:03.407756 16916 heartbeater.cc:347] Connected to a master server at 127.5.222.254:32975
I20260809 04:41:03.407989 16916 heartbeater.cc:464] Registering TS with master...
I20260809 04:41:03.408568 16916 heartbeater.cc:511] Master 127.5.222.254:32975 requested a full tablet report, sending...
I20260809 04:41:03.410356 16700 ts_manager.cc:194] Registered new tserver with Master: 72202a8707a447f0bb28ccacfcf4ffa3 (127.5.222.193:42789)
I20260809 04:41:03.412001 16700 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.222.193:48709
W20260809 04:41:03.514201 16919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:03.514472 16919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:03.514510 16919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:03.518493 16919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:03.518553 16919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:03.518607 16919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:41:03.523265 16919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:32975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:03.524447 16919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:03.525821 16919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:03.535720 16924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:03.535713 16925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:03.535856 16927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:03.535985 16919 server_base.cc:1034] running on GCE node
I20260809 04:41:03.536379 16919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:03.537035 16919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:03.538225 16919 hybrid_clock.cc:648] HybridClock initialized: now 1786250463538196 us; error 42 us; skew 500 ppm
I20260809 04:41:03.540186 16919 webserver.cc:460] Webserver started at http://127.5.222.194:43167/ using document root <none> and password file <none>
I20260809 04:41:03.540791 16919 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:03.540889 16919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:03.541245 16919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:41:03.543378 16919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "bc8310999b9940c3b25118c3b3d23bf3"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "f62ea3c3109867e3b58ba4b211abd308"
server_key_iv: "ce3112a7b5c7fa7774f047ed806a7059"
server_key_version: "encryptionkey@0"
I20260809 04:41:03.544072 16919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "bc8310999b9940c3b25118c3b3d23bf3"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "f62ea3c3109867e3b58ba4b211abd308"
server_key_iv: "ce3112a7b5c7fa7774f047ed806a7059"
server_key_version: "encryptionkey@0"
I20260809 04:41:03.547921 16919 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 04:41:03.550666 16933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:03.551998 16919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:41:03.552141 16919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "bc8310999b9940c3b25118c3b3d23bf3"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "f62ea3c3109867e3b58ba4b211abd308"
server_key_iv: "ce3112a7b5c7fa7774f047ed806a7059"
server_key_version: "encryptionkey@0"
I20260809 04:41:03.552284 16919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:03.573014 16919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:03.573822 16919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:03.574038 16919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:03.574741 16919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:41:03.575739 16919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:41:03.575811 16919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:03.575886 16919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:41:03.575927 16919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:03.584745 16919 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:41017
I20260809 04:41:03.584775 17061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:41017 every 8 connection(s)
I20260809 04:41:03.585726 16919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:41:03.594952 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 16919
I20260809 04:41:03.595095 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:41:03.595225 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:03.587670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:45949 (local address 127.5.222.254:32975)
0809 04:41:03.587818 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:03.587822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:03.588854 (+ 1032us) server_negotiation.cc:407] Connection header received
0809 04:41:03.590374 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:03.590378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:03.590440 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:03.590551 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:03.591472 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.592011 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.593203 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.593433 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.593552 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:03.594632 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:03.594636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:03.594638 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:41:03.594654 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:03.594679 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:41:03.594691 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:03.594770 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:03.594978 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 04:41:03.595053 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:41:03.595465 6011 external_mini_cluster.cc:1428] Setting key dc0489e93ab24dc99fa18e983b81f922
I20260809 04:41:03.596344 17064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:03.588020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:32975 (local address 127.5.222.194:45949)
0809 04:41:03.588685 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:03.588727 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:03.590130 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:03.590731 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:03.590738 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:03.590821 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:03.591314 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:03.591325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.592143 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.592146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:03.593041 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:03.593051 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.593597 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.593601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:03.593783 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:03.594356 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:41:03.594358 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:03.594400 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:41:03.594464 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:41:03.594799 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:41:03.594816 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:41:03.594821 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:41:03.594938 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 04:41:03.595253 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":114,"threads_started":1}
I20260809 04:41:03.597615 17062 heartbeater.cc:347] Connected to a master server at 127.5.222.254:32975
I20260809 04:41:03.597769 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:32975
--builtin_ntp_servers=127.5.222.212:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 04:41:03.597891 17062 heartbeater.cc:464] Registering TS with master...
I20260809 04:41:03.598532 17062 heartbeater.cc:511] Master 127.5.222.254:32975 requested a full tablet report, sending...
I20260809 04:41:03.599835 16700 ts_manager.cc:194] Registered new tserver with Master: bc8310999b9940c3b25118c3b3d23bf3 (127.5.222.194:41017)
I20260809 04:41:03.600410 16700 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.222.194:45949
W20260809 04:41:03.713629 17065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:03.713905 17065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:03.713968 17065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:03.718065 17065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:03.718129 17065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:03.718227 17065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:41:03.723006 17065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:32975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:03.724146 17065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:03.725641 17065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:03.735245 17074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:03.735265 17071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:03.735263 17070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:03.735319 17065 server_base.cc:1034] running on GCE node
I20260809 04:41:03.735906 17065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:03.736584 17065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:03.737767 17065 hybrid_clock.cc:648] HybridClock initialized: now 1786250463737749 us; error 29 us; skew 500 ppm
I20260809 04:41:03.739526 17065 webserver.cc:460] Webserver started at http://127.5.222.195:42633/ using document root <none> and password file <none>
I20260809 04:41:03.740103 17065 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:03.740195 17065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:03.740435 17065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:41:03.742585 17065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "b09b40d2d9ad46b0836325a6840a9ffa"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "21dac4acd5227bd4d36197d85a6390c2"
server_key_iv: "4113b9220e53f3ae729c36e37d1e6131"
server_key_version: "encryptionkey@0"
I20260809 04:41:03.743146 17065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "b09b40d2d9ad46b0836325a6840a9ffa"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "21dac4acd5227bd4d36197d85a6390c2"
server_key_iv: "4113b9220e53f3ae729c36e37d1e6131"
server_key_version: "encryptionkey@0"
I20260809 04:41:03.746866 17065 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:41:03.749396 17080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:03.750669 17065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:41:03.750809 17065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "b09b40d2d9ad46b0836325a6840a9ffa"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "21dac4acd5227bd4d36197d85a6390c2"
server_key_iv: "4113b9220e53f3ae729c36e37d1e6131"
server_key_version: "encryptionkey@0"
I20260809 04:41:03.750926 17065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:03.767323 17065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:03.768038 17065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:03.768246 17065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:03.768855 17065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:41:03.769865 17065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:41:03.769938 17065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:03.770010 17065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:41:03.770066 17065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:03.778918 17065 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:40527
I20260809 04:41:03.778946 17205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:40527 every 8 connection(s)
I20260809 04:41:03.779824 17065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:41:03.785533 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 17065
I20260809 04:41:03.785662 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:41:03.786000 6011 external_mini_cluster.cc:1428] Setting key 0bf0ee86ff0851fef94bbdf27049bae8
I20260809 04:41:03.789532 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:03.781929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:52829 (local address 127.5.222.254:32975)
0809 04:41:03.782139 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:03.782143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:03.783139 (+ 996us) server_negotiation.cc:407] Connection header received
0809 04:41:03.784480 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:03.784482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:03.784530 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:03.784611 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:03.785712 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.786328 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.787532 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.787765 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.787882 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:03.788989 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:03.788993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:03.788995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:41:03.789015 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:03.789033 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:41:03.789041 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:03.789118 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:03.789326 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 04:41:03.789388 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 04:41:03.790380 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:03.782300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:32975 (local address 127.5.222.195:52829)
0809 04:41:03.782975 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:03.783017 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:03.784258 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:03.784772 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:03.784778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:03.784891 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:03.785537 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:03.785559 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.786495 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.786503 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:03.787380 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:03.787391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:03.787925 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:03.787930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:03.788141 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:03.788724 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:41:03.788725 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:03.788765 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:41:03.788831 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:41:03.789132 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:41:03.789141 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:41:03.789170 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 04:41:03.789314 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 04:41:03.789609 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":130,"threads_started":1}
I20260809 04:41:03.791428 17206 heartbeater.cc:347] Connected to a master server at 127.5.222.254:32975
I20260809 04:41:03.791667 17206 heartbeater.cc:464] Registering TS with master...
I20260809 04:41:03.792182 17206 heartbeater.cc:511] Master 127.5.222.254:32975 requested a full tablet report, sending...
I20260809 04:41:03.793226 16700 ts_manager.cc:194] Registered new tserver with Master: b09b40d2d9ad46b0836325a6840a9ffa (127.5.222.195:40527)
I20260809 04:41:03.793787 16700 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.222.195:52829
I20260809 04:41:03.799844 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 04:41:03.802709 16744 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:41:03.801837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47974 (local address 127.5.222.254:32975)
0809 04:41:03.801961 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:03.801966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:03.802122 (+ 156us) server_negotiation.cc:407] Connection header received
0809 04:41:03.802219 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:03.802221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:03.802259 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:03.802330 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:03.802552 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47974: BlockingRecv error: recv got EOF from 127.0.0.1:47974 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:41:03.803097 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 16767
I20260809 04:41:03.813369 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 16919
I20260809 04:41:03.820676 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 17065
I20260809 04:41:03.829083 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 16662
2026-08-09T04:41:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (907 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T04:41:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:41:03Z Disabled control of system clock
I20260809 04:41:03.860766 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:35485
--webserver_interface=127.5.222.254
--webserver_port=0
--builtin_ntp_servers=127.5.222.212:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:35485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 04:41:03.975751 17219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:03.976011 17219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:03.976046 17219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:03.980031 17219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:41:03.980095 17219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:03.980111 17219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:03.980129 17219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:41:03.980144 17219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:41:03.984859 17219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:35485
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:35485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:03.986173 17219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:03.987622 17219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:03.996502 17226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:03.996483 17225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:03.996464 17228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:03.997359 17219 server_base.cc:1034] running on GCE node
I20260809 04:41:03.997927 17219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:03.999027 17219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:04.000190 17219 hybrid_clock.cc:648] HybridClock initialized: now 1786250464000158 us; error 47 us; skew 500 ppm
I20260809 04:41:04.002115 17219 webserver.cc:460] Webserver started at http://127.5.222.254:41975/ using document root <none> and password file <none>
I20260809 04:41:04.002702 17219 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:04.002765 17219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:04.002993 17219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:41:04.005072 17219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "ae4f18de63b8426584beb755f008bff8"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "5377fb7a0721c0b8f74006be84dcdff4"
server_key_iv: "dd61e06846de40b248718dbcadf1bda2"
server_key_version: "encryptionkey@0"
I20260809 04:41:04.005704 17219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "ae4f18de63b8426584beb755f008bff8"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "5377fb7a0721c0b8f74006be84dcdff4"
server_key_iv: "dd61e06846de40b248718dbcadf1bda2"
server_key_version: "encryptionkey@0"
I20260809 04:41:04.009539 17219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:41:04.011996 17234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:04.013351 17219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:41:04.013499 17219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "ae4f18de63b8426584beb755f008bff8"
format_stamp: "Formatted at 2026-08-09 04:41:03 on dist-test-slave-vxj2"
server_key: "5377fb7a0721c0b8f74006be84dcdff4"
server_key_iv: "dd61e06846de40b248718dbcadf1bda2"
server_key_version: "encryptionkey@0"
I20260809 04:41:04.013620 17219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:04.018481 17219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:04.019145 17219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:04.019343 17219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:04.026633 17219 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:35485
I20260809 04:41:04.026654 17292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:35485 every 8 connection(s)
I20260809 04:41:04.027581 17219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:41:04.030758 17293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:41:04.036561 17293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8: Bootstrap starting.
I20260809 04:41:04.036895 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 17219
I20260809 04:41:04.037014 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:41:04.037343 6011 external_mini_cluster.cc:1428] Setting key 795dd1502d0bea92dd6a2c94aef6f5de
I20260809 04:41:04.039446 17293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8: Neither blocks nor log segments found. Creating new log.
I20260809 04:41:04.040371 17293 log.cc:824] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:41:04.042718 17293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8: No bootstrap required, opened a new log
I20260809 04:41:04.043612 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:04.038787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45446 (local address 127.5.222.254:35485)
0809 04:41:04.039559 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:04.039583 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:04.039631 (+ 48us) server_negotiation.cc:407] Connection header received
0809 04:41:04.040541 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:04.040565 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:04.040985 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:04.041367 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:04.042135 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:04.042140 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:04.042155 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:41:04.042167 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:04.042233 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:41:04.042255 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:04.042390 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:04.042460 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 04:41:04.042700 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":130,"threads_started":1}
I20260809 04:41:04.046907 17293 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4f18de63b8426584beb755f008bff8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35485 } }
I20260809 04:41:04.047119 17293 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:41:04.047152 17293 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4f18de63b8426584beb755f008bff8, State: Initialized, Role: FOLLOWER
I20260809 04:41:04.047498 17293 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4f18de63b8426584beb755f008bff8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35485 } }
I20260809 04:41:04.047605 17293 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:41:04.047688 17293 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:41:04.047801 17293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:41:04.048722 17293 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4f18de63b8426584beb755f008bff8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35485 } }
I20260809 04:41:04.049078 17293 leader_election.cc:302] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae4f18de63b8426584beb755f008bff8; no voters:
I20260809 04:41:04.049443 17293 leader_election.cc:288] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:41:04.049522 17298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:41:04.049700 17298 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [term 1 LEADER]: Becoming Leader. State: Replica: ae4f18de63b8426584beb755f008bff8, State: Running, Role: LEADER
I20260809 04:41:04.049983 17298 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4f18de63b8426584beb755f008bff8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35485 } }
I20260809 04:41:04.050719 17293 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:41:04.051700 17299 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae4f18de63b8426584beb755f008bff8. Latest consensus state: current_term: 1 leader_uuid: "ae4f18de63b8426584beb755f008bff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4f18de63b8426584beb755f008bff8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35485 } } }
I20260809 04:41:04.051831 17299 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:41:04.052109 17301 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae4f18de63b8426584beb755f008bff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4f18de63b8426584beb755f008bff8" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35485 } } }
I20260809 04:41:04.052188 17301 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:41:04.052841 17308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:41:04.055517 17308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:41:04.060811 17308 catalog_manager.cc:1347] Generated new cluster ID: f2968aa53b0f4770a7605d4090433cf2
I20260809 04:41:04.060899 17308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:41:04.087337 17308 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:41:04.087705 17308 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:41:04.098690 17308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae4f18de63b8426584beb755f008bff8: Generated new TSK 0
I20260809 04:41:04.099398 17308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:41:04.110707 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35485
--builtin_ntp_servers=127.5.222.212:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 04:41:04.227721 17318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:04.228015 17318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:04.228057 17318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:04.232188 17318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:04.232260 17318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:04.232318 17318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:41:04.237308 17318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:04.238391 17318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:04.239750 17318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:04.249728 17326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:04.249840 17324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:04.249742 17323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:04.249939 17318 server_base.cc:1034] running on GCE node
I20260809 04:41:04.250573 17318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:04.251449 17318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:04.252641 17318 hybrid_clock.cc:648] HybridClock initialized: now 1786250464252627 us; error 34 us; skew 500 ppm
I20260809 04:41:04.254664 17318 webserver.cc:460] Webserver started at http://127.5.222.193:37151/ using document root <none> and password file <none>
I20260809 04:41:04.255282 17318 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:04.255378 17318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:04.255616 17318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:41:04.257870 17318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "d7d90ba89899415b836632d24ee525b8"
format_stamp: "Formatted at 2026-08-09 04:41:04 on dist-test-slave-vxj2"
server_key: "0d87a0e37c87681198d178445d56f167"
server_key_iv: "601ee4557d93a8d7d42a80dc2f13fcc3"
server_key_version: "encryptionkey@0"
I20260809 04:41:04.258458 17318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "d7d90ba89899415b836632d24ee525b8"
format_stamp: "Formatted at 2026-08-09 04:41:04 on dist-test-slave-vxj2"
server_key: "0d87a0e37c87681198d178445d56f167"
server_key_iv: "601ee4557d93a8d7d42a80dc2f13fcc3"
server_key_version: "encryptionkey@0"
I20260809 04:41:04.262176 17318 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:41:04.264632 17334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:04.265966 17318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:41:04.266114 17318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "d7d90ba89899415b836632d24ee525b8"
format_stamp: "Formatted at 2026-08-09 04:41:04 on dist-test-slave-vxj2"
server_key: "0d87a0e37c87681198d178445d56f167"
server_key_iv: "601ee4557d93a8d7d42a80dc2f13fcc3"
server_key_version: "encryptionkey@0"
I20260809 04:41:04.266234 17318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:04.270936 17318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:04.271615 17318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:04.271780 17318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:04.272257 17318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:41:04.273130 17318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:41:04.273264 17318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:04.273307 17318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:41:04.273329 17318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:04.282202 17318 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:36607
I20260809 04:41:04.282236 17461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:36607 every 8 connection(s)
I20260809 04:41:04.283073 17318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:41:04.288355 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 17318
I20260809 04:41:04.288482 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:41:04.288857 6011 external_mini_cluster.cc:1428] Setting key 27ad8ac956ad423bb2fb526e777cdb4d
I20260809 04:41:04.289263 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:04.284849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:41007 (local address 127.5.222.254:35485)
0809 04:41:04.285007 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:04.285010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:04.286022 (+ 1012us) server_negotiation.cc:407] Connection header received
0809 04:41:04.287386 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:04.287389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:04.287440 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:04.287555 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:04.288750 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:04.288753 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:04.288755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:41:04.288768 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:04.288790 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:41:04.288799 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:04.288885 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:04.289051 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 04:41:04.289091 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:41:04.290164 17464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:04.285214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:35485 (local address 127.5.222.193:41007)
0809 04:41:04.285869 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:04.285910 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:04.287146 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:04.287725 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:04.287731 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:04.287828 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:04.288415 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:41:04.288418 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:04.288479 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:41:04.288571 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:41:04.288900 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:41:04.288910 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:41:04.288916 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:41:04.289098 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 04:41:04.289412 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":121,"threads_started":1}
I20260809 04:41:04.290926 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35485
--builtin_ntp_servers=127.5.222.212:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 04:41:04.291458 17462 heartbeater.cc:347] Connected to a master server at 127.5.222.254:35485
I20260809 04:41:04.291703 17462 heartbeater.cc:464] Registering TS with master...
I20260809 04:41:04.292186 17462 heartbeater.cc:511] Master 127.5.222.254:35485 requested a full tablet report, sending...
I20260809 04:41:04.294055 17255 ts_manager.cc:194] Registered new tserver with Master: d7d90ba89899415b836632d24ee525b8 (127.5.222.193:36607)
W20260809 04:41:04.403183 17465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:04.403426 17465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:04.403465 17465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:04.407471 17465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:04.407524 17465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:04.407579 17465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:41:04.412344 17465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:04.413422 17465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:04.414789 17465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:04.424690 17475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:04.424710 17470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:04.424690 17472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:04.424897 17465 server_base.cc:1034] running on GCE node
I20260809 04:41:04.425464 17465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:04.426083 17465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:04.427244 17465 hybrid_clock.cc:648] HybridClock initialized: now 1786250464427214 us; error 42 us; skew 500 ppm
I20260809 04:41:04.428964 17465 webserver.cc:460] Webserver started at http://127.5.222.194:40657/ using document root <none> and password file <none>
I20260809 04:41:04.429579 17465 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:04.429641 17465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:04.429890 17465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:41:04.431861 17465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "92a18fb17dc14d9aa92e1c9b886a8ab7"
format_stamp: "Formatted at 2026-08-09 04:41:04 on dist-test-slave-vxj2"
server_key: "1e31ee2811fb1c1200085cba4da49857"
server_key_iv: "c0ef1b02abc0b4e246e0002f1d5d4ff1"
server_key_version: "encryptionkey@0"
I20260809 04:41:04.432452 17465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "92a18fb17dc14d9aa92e1c9b886a8ab7"
format_stamp: "Formatted at 2026-08-09 04:41:04 on dist-test-slave-vxj2"
server_key: "1e31ee2811fb1c1200085cba4da49857"
server_key_iv: "c0ef1b02abc0b4e246e0002f1d5d4ff1"
server_key_version: "encryptionkey@0"
I20260809 04:41:04.436225 17465 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.001s
I20260809 04:41:04.438737 17482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:04.439890 17465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:41:04.440016 17465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "92a18fb17dc14d9aa92e1c9b886a8ab7"
format_stamp: "Formatted at 2026-08-09 04:41:04 on dist-test-slave-vxj2"
server_key: "1e31ee2811fb1c1200085cba4da49857"
server_key_iv: "c0ef1b02abc0b4e246e0002f1d5d4ff1"
server_key_version: "encryptionkey@0"
I20260809 04:41:04.440122 17465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:04.444554 17465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:04.445123 17465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:04.445361 17465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:04.445945 17465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:41:04.446812 17465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:41:04.446882 17465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:04.446954 17465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:41:04.447007 17465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:04.456470 17465 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:46559
I20260809 04:41:04.456504 17606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:46559 every 8 connection(s)
I20260809 04:41:04.457444 17465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:41:04.458079 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 17465
I20260809 04:41:04.458242 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:41:04.458599 6011 external_mini_cluster.cc:1428] Setting key 341bc4023bd136382a227690678eb27d
I20260809 04:41:04.460933 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35485
--builtin_ntp_servers=127.5.222.212:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 04:41:04.464294 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:04.459385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:51665 (local address 127.5.222.254:35485)
0809 04:41:04.459543 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:04.459546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:04.460833 (+ 1287us) server_negotiation.cc:407] Connection header received
0809 04:41:04.462282 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:04.462289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:04.462343 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:04.462458 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:04.463817 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:04.463821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:04.463824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:41:04.463836 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:04.463860 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:41:04.463869 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:04.463956 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:04.464132 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 04:41:04.464173 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:41:04.465430 17609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:04.459904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:35485 (local address 127.5.222.194:51665)
0809 04:41:04.460640 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:04.460690 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:04.462008 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:04.462652 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:04.462663 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:04.462783 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:04.463454 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:41:04.463456 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:04.463513 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:41:04.463604 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:41:04.463967 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:41:04.463978 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:41:04.463987 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:41:04.464162 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 04:41:04.464388 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":137,"threads_started":1}
I20260809 04:41:04.466703 17607 heartbeater.cc:347] Connected to a master server at 127.5.222.254:35485
I20260809 04:41:04.466950 17607 heartbeater.cc:464] Registering TS with master...
I20260809 04:41:04.467377 17607 heartbeater.cc:511] Master 127.5.222.254:35485 requested a full tablet report, sending...
I20260809 04:41:04.468572 17255 ts_manager.cc:194] Registered new tserver with Master: 92a18fb17dc14d9aa92e1c9b886a8ab7 (127.5.222.194:46559)
W20260809 04:41:04.576696 17610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:04.577000 17610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:04.577075 17610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:04.581081 17610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:04.581192 17610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:04.581280 17610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:41:04.586131 17610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:04.587320 17610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:04.588750 17610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:04.598492 17618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:04.598536 17616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:04.598605 17615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:04.598932 17610 server_base.cc:1034] running on GCE node
I20260809 04:41:04.599556 17610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:04.600287 17610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:04.601482 17610 hybrid_clock.cc:648] HybridClock initialized: now 1786250464601465 us; error 38 us; skew 500 ppm
I20260809 04:41:04.603513 17610 webserver.cc:460] Webserver started at http://127.5.222.195:38215/ using document root <none> and password file <none>
I20260809 04:41:04.604125 17610 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:04.604223 17610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:04.604468 17610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:41:04.606695 17610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "08268fee88004adda98a2211dab487b5"
format_stamp: "Formatted at 2026-08-09 04:41:04 on dist-test-slave-vxj2"
server_key: "4ca31c1037c52e91b7e5755ddd6b0154"
server_key_iv: "98f6cefde0b40c67c2e819596e5b3861"
server_key_version: "encryptionkey@0"
I20260809 04:41:04.607291 17610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "08268fee88004adda98a2211dab487b5"
format_stamp: "Formatted at 2026-08-09 04:41:04 on dist-test-slave-vxj2"
server_key: "4ca31c1037c52e91b7e5755ddd6b0154"
server_key_iv: "98f6cefde0b40c67c2e819596e5b3861"
server_key_version: "encryptionkey@0"
I20260809 04:41:04.611454 17610 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:41:04.614101 17624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:04.615307 17610 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:41:04.615450 17610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "08268fee88004adda98a2211dab487b5"
format_stamp: "Formatted at 2026-08-09 04:41:04 on dist-test-slave-vxj2"
server_key: "4ca31c1037c52e91b7e5755ddd6b0154"
server_key_iv: "98f6cefde0b40c67c2e819596e5b3861"
server_key_version: "encryptionkey@0"
I20260809 04:41:04.615566 17610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:04.620483 17610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:04.621238 17610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:04.621387 17610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:04.621886 17610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:41:04.622738 17610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:41:04.622794 17610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:04.622833 17610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:41:04.622854 17610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:04.631832 17610 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:33195
I20260809 04:41:04.631855 17751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:33195 every 8 connection(s)
I20260809 04:41:04.632738 17610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:41:04.638424 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 17610
I20260809 04:41:04.638549 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:41:04.638887 6011 external_mini_cluster.cc:1428] Setting key 6689363a1def04bb9dcf5f77f7412b7e
I20260809 04:41:04.639022 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:04.634660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:45399 (local address 127.5.222.254:35485)
0809 04:41:04.634814 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:04.634818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:04.635911 (+ 1093us) server_negotiation.cc:407] Connection header received
0809 04:41:04.637209 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:04.637212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:04.637250 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:04.637330 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:04.638405 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:04.638408 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:04.638412 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:41:04.638426 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:04.638455 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:41:04.638467 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:04.638574 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:04.638828 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 04:41:04.638877 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:41:04.639860 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:04.635050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:35485 (local address 127.5.222.195:45399)
0809 04:41:04.635747 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:04.635810 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:04.636974 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:04.637498 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:04.637505 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:04.637589 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:04.638118 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:41:04.638119 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:04.638159 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:41:04.638224 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:41:04.638595 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:41:04.638607 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:41:04.638613 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:41:04.638786 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 04:41:04.639016 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":114,"threads_started":1}
I20260809 04:41:04.640990 17752 heartbeater.cc:347] Connected to a master server at 127.5.222.254:35485
I20260809 04:41:04.641278 17752 heartbeater.cc:464] Registering TS with master...
I20260809 04:41:04.641709 17752 heartbeater.cc:511] Master 127.5.222.254:35485 requested a full tablet report, sending...
I20260809 04:41:04.642638 17254 ts_manager.cc:194] Registered new tserver with Master: 08268fee88004adda98a2211dab487b5 (127.5.222.195:33195)
I20260809 04:41:04.653329 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 04:41:04.656059 17296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:41:04.655246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58098 (local address 127.5.222.254:35485)
0809 04:41:04.655375 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:04.655379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:04.655529 (+ 150us) server_negotiation.cc:407] Connection header received
0809 04:41:04.655605 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:04.655607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:04.655642 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:04.655715 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:04.655927 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58098: BlockingRecv error: recv got EOF from 127.0.0.1:58098 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:41:04.656244 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 17318
I20260809 04:41:04.667029 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 17465
I20260809 04:41:04.673763 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 17610
I20260809 04:41:04.681895 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 17219
2026-08-09T04:41:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (852 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:41:04 dist-test-slave-vxj2 krb5kdc[17763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:41:04 dist-test-slave-vxj2 krb5kdc[17763](info): set up 2 sockets
Aug 09 04:41:04 dist-test-slave-vxj2 krb5kdc[17763](info): commencing operation
krb5kdc: starting...
W20260809 04:41:05.833992 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:41:05 dist-test-slave-vxj2 krb5kdc[17763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:41:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:41:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:41:05.960035 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:39901
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:39901
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:41:06.074684 17777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:06.074944 17777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:06.074985 17777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:06.079051 17777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:41:06.079107 17777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:06.079128 17777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:06.079144 17777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:41:06.079160 17777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:41:06.084208 17777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:39901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:39901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:06.085142 17777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:06.086208 17777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:06.094520 17782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:06.094637 17786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:06.094729 17783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:06.094974 17777 server_base.cc:1034] running on GCE node
I20260809 04:41:06.095602 17777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:06.096707 17777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:06.097918 17777 hybrid_clock.cc:648] HybridClock initialized: now 1786250466097890 us; error 37 us; skew 500 ppm
Aug 09 04:41:06 dist-test-slave-vxj2 krb5kdc[17763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250466, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:41:06.105674 17777 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:41:06.106652 17777 webserver.cc:460] Webserver started at http://127.5.222.254:37951/ using document root <none> and password file <none>
I20260809 04:41:06.107259 17777 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:06.107352 17777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:06.107570 17777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:41:06.109758 17777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "7e4ab1c226ff467f820f22dcafaf963e"
format_stamp: "Formatted at 2026-08-09 04:41:06 on dist-test-slave-vxj2"
server_key: "9d0eaf371a5fa61cf8ce5bb24f7ab41b"
server_key_iv: "3edfb3725f3fa3627e2b25f616cbfcf8"
server_key_version: "encryptionkey@0"
I20260809 04:41:06.110463 17777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "7e4ab1c226ff467f820f22dcafaf963e"
format_stamp: "Formatted at 2026-08-09 04:41:06 on dist-test-slave-vxj2"
server_key: "9d0eaf371a5fa61cf8ce5bb24f7ab41b"
server_key_iv: "3edfb3725f3fa3627e2b25f616cbfcf8"
server_key_version: "encryptionkey@0"
I20260809 04:41:06.114318 17777 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 04:41:06.116802 17793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:06.118079 17777 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:41:06.118232 17777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "7e4ab1c226ff467f820f22dcafaf963e"
format_stamp: "Formatted at 2026-08-09 04:41:06 on dist-test-slave-vxj2"
server_key: "9d0eaf371a5fa61cf8ce5bb24f7ab41b"
server_key_iv: "3edfb3725f3fa3627e2b25f616cbfcf8"
server_key_version: "encryptionkey@0"
I20260809 04:41:06.118345 17777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:06.131448 17777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:06.134443 17777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:06.134647 17777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:06.142128 17777 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:39901
I20260809 04:41:06.142416 17854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:39901 every 8 connection(s)
I20260809 04:41:06.143129 17777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:41:06.145907 17855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:41:06.146373 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 17777
I20260809 04:41:06.146535 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:41:06.146837 6011 external_mini_cluster.cc:1428] Setting key b724851d30758c36d2e4719865509e31
I20260809 04:41:06.151782 17855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e: Bootstrap starting.
I20260809 04:41:06.154148 17855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e: Neither blocks nor log segments found. Creating new log.
I20260809 04:41:06.154906 17855 log.cc:824] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:41:06 dist-test-slave-vxj2 krb5kdc[17763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:41:06.157256 17855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e: No bootstrap required, opened a new log
I20260809 04:41:06.161463 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.148246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51960 (local address 127.5.222.254:39901)
0809 04:41:06.148680 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:06.148694 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:06.148731 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:41:06.149463 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:06.149482 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:06.149797 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:06.150151 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:06.151012 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.152039 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.152906 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.153460 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.153636 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.156453 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:06.156458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:06.156472 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:41:06.156498 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:06.158218 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:06.158717 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:06.158722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:06.158726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:06.158834 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:06.159187 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:06.159190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:06.159191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:06.159593 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:06.159766 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:06.160123 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 04:41:06.160522 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":130,"threads_started":1}
I20260809 04:41:06.161463 17855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4ab1c226ff467f820f22dcafaf963e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39901 } }
I20260809 04:41:06.161743 17855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:41:06.161819 17855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e4ab1c226ff467f820f22dcafaf963e, State: Initialized, Role: FOLLOWER
I20260809 04:41:06.162274 17855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4ab1c226ff467f820f22dcafaf963e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39901 } }
I20260809 04:41:06.162400 17855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:41:06.162514 17855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:41:06.162655 17855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:41:06.163849 17855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4ab1c226ff467f820f22dcafaf963e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39901 } }
I20260809 04:41:06.164196 17855 leader_election.cc:302] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e4ab1c226ff467f820f22dcafaf963e; no voters:
I20260809 04:41:06.164506 17855 leader_election.cc:288] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:41:06.164628 17860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:41:06.164906 17860 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [term 1 LEADER]: Becoming Leader. State: Replica: 7e4ab1c226ff467f820f22dcafaf963e, State: Running, Role: LEADER
I20260809 04:41:06.165311 17860 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4ab1c226ff467f820f22dcafaf963e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39901 } }
I20260809 04:41:06.165804 17855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:41:06.166932 17862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e4ab1c226ff467f820f22dcafaf963e. Latest consensus state: current_term: 1 leader_uuid: "7e4ab1c226ff467f820f22dcafaf963e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4ab1c226ff467f820f22dcafaf963e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39901 } } }
I20260809 04:41:06.167083 17862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [sys.catalog]: This master's current role is: LEADER
I20260809 04:41:06.167476 17870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:41:06.167467 17861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e4ab1c226ff467f820f22dcafaf963e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4ab1c226ff467f820f22dcafaf963e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39901 } } }
I20260809 04:41:06.167572 17861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e [sys.catalog]: This master's current role is: LEADER
I20260809 04:41:06.170605 17870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:41:06.176373 17870 catalog_manager.cc:1347] Generated new cluster ID: b8e8ec1ad3834aa5a5fefff8d584febe
I20260809 04:41:06.176453 17870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:41:06.185123 17870 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:41:06.186048 17870 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:41:06.190063 17870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e: Generated new TSK 0
I20260809 04:41:06.190869 17870 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:41:06.229419 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:39901
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:41:06.343418 17884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:06.343667 17884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:06.343709 17884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:06.347914 17884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:06.347975 17884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:06.348031 17884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:41:06.353340 17884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:39901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:06.354368 17884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:06.355443 17884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:06.364897 17890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:06.364908 17889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:06.364840 17892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:06.364984 17884 server_base.cc:1034] running on GCE node
I20260809 04:41:06.365513 17884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:06.366209 17884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:06.367411 17884 hybrid_clock.cc:648] HybridClock initialized: now 1786250466367381 us; error 36 us; skew 500 ppm
Aug 09 04:41:06 dist-test-slave-vxj2 krb5kdc[17763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250466, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:41:06.370386 17884 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:41:06.371214 17884 webserver.cc:460] Webserver started at http://127.5.222.193:42053/ using document root <none> and password file <none>
I20260809 04:41:06.371762 17884 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:06.371822 17884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:06.372046 17884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:41:06.374230 17884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "4c22a8821d284adc909c9b1c9ef19e67"
format_stamp: "Formatted at 2026-08-09 04:41:06 on dist-test-slave-vxj2"
server_key: "1a59d5e484e0e7bef1920976b5858ca9"
server_key_iv: "1990d63a7950c4c5a5849059a1e23575"
server_key_version: "encryptionkey@0"
I20260809 04:41:06.374877 17884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "4c22a8821d284adc909c9b1c9ef19e67"
format_stamp: "Formatted at 2026-08-09 04:41:06 on dist-test-slave-vxj2"
server_key: "1a59d5e484e0e7bef1920976b5858ca9"
server_key_iv: "1990d63a7950c4c5a5849059a1e23575"
server_key_version: "encryptionkey@0"
I20260809 04:41:06.378756 17884 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:41:06.381475 17900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:06.382712 17884 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 04:41:06.382992 17884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "4c22a8821d284adc909c9b1c9ef19e67"
format_stamp: "Formatted at 2026-08-09 04:41:06 on dist-test-slave-vxj2"
server_key: "1a59d5e484e0e7bef1920976b5858ca9"
server_key_iv: "1990d63a7950c4c5a5849059a1e23575"
server_key_version: "encryptionkey@0"
I20260809 04:41:06.383127 17884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:06.396957 17884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:06.400204 17884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:06.400419 17884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:06.400991 17884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:41:06.401826 17884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:41:06.401875 17884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:06.401944 17884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:41:06.401978 17884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:06.411468 17884 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:33607
I20260809 04:41:06.411496 18027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:33607 every 8 connection(s)
I20260809 04:41:06.412307 17884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:41:06.416118 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 17884
I20260809 04:41:06.416261 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:41:06.416604 6011 external_mini_cluster.cc:1428] Setting key 3073ffceaecacd94dbb8235c9fafa683
Aug 09 04:41:06 dist-test-slave-vxj2 krb5kdc[17763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250466, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:41:06.427819 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.414215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:48373 (local address 127.5.222.254:39901)
0809 04:41:06.414362 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:06.414366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:06.415103 (+ 737us) server_negotiation.cc:407] Connection header received
0809 04:41:06.416855 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:06.416858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:06.416909 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:06.416976 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:06.418233 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.418756 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.420432 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.420680 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.420810 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.423694 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:06.423699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:06.423702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:41:06.423738 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:06.425532 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:06.426052 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:06.426057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:06.426060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:06.426166 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:06.426529 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:06.426533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:06.426535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:06.426797 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:06.426922 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:06.427393 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 04:41:06.427547 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
I20260809 04:41:06.428368 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.414441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:39901 (local address 127.5.222.193:48373)
0809 04:41:06.414932 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:06.414975 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:06.416582 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:06.417229 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:06.417237 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:06.417586 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:06.418045 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.418059 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.418897 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.418904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.420288 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.420299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.420845 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.420850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.421024 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.421744 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:06.421772 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:06.423518 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:41:06.425666 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:41:06.425671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:41:06.425681 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:41:06.425944 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:41:06.426278 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:41:06.426280 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:41:06.426282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:41:06.426403 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:41:06.426921 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:41:06.426926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:41:06.427137 (+ 211us) client_negotiation.cc:769] Sending connection context
0809 04:41:06.427287 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 04:41:06.427521 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":110,"threads_started":1}
I20260809 04:41:06.429639 18028 heartbeater.cc:347] Connected to a master server at 127.5.222.254:39901
I20260809 04:41:06.429862 18028 heartbeater.cc:464] Registering TS with master...
I20260809 04:41:06.430399 18028 heartbeater.cc:511] Master 127.5.222.254:39901 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:41:06.431919 17817 ts_manager.cc:194] Registered new tserver with Master: 4c22a8821d284adc909c9b1c9ef19e67 (127.5.222.193:33607)
I20260809 04:41:06.433043 17817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:48373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:41:06.448246 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:39901
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:41:06.559682 18033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:06.559938 18033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:06.559985 18033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:06.564328 18033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:06.564384 18033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:06.564441 18033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:41:06.570003 18033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:39901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:06.571084 18033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:06.572134 18033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:06.581743 18041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:06.581765 18038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:06.581768 18039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:06.581852 18033 server_base.cc:1034] running on GCE node
I20260809 04:41:06.582412 18033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:06.583122 18033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:06.584321 18033 hybrid_clock.cc:648] HybridClock initialized: now 1786250466584281 us; error 51 us; skew 500 ppm
Aug 09 04:41:06 dist-test-slave-vxj2 krb5kdc[17763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250466, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:41:06.587146 18033 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:41:06.588053 18033 webserver.cc:460] Webserver started at http://127.5.222.194:34079/ using document root <none> and password file <none>
I20260809 04:41:06.588605 18033 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:06.588697 18033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:06.588960 18033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:41:06.591027 18033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "1486b9470cf742c5915a708c1efa06bf"
format_stamp: "Formatted at 2026-08-09 04:41:06 on dist-test-slave-vxj2"
server_key: "0977034f3e2ef99a1806d74dbb4e67cd"
server_key_iv: "6ad22d4ee252e03bd8b8e85356500be5"
server_key_version: "encryptionkey@0"
I20260809 04:41:06.591590 18033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "1486b9470cf742c5915a708c1efa06bf"
format_stamp: "Formatted at 2026-08-09 04:41:06 on dist-test-slave-vxj2"
server_key: "0977034f3e2ef99a1806d74dbb4e67cd"
server_key_iv: "6ad22d4ee252e03bd8b8e85356500be5"
server_key_version: "encryptionkey@0"
I20260809 04:41:06.595335 18033 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:41:06.597599 18048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:06.598707 18033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:41:06.598855 18033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "1486b9470cf742c5915a708c1efa06bf"
format_stamp: "Formatted at 2026-08-09 04:41:06 on dist-test-slave-vxj2"
server_key: "0977034f3e2ef99a1806d74dbb4e67cd"
server_key_iv: "6ad22d4ee252e03bd8b8e85356500be5"
server_key_version: "encryptionkey@0"
I20260809 04:41:06.598964 18033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:06.614295 18033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:06.617251 18033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:06.617444 18033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:06.618031 18033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:41:06.618854 18033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:41:06.618924 18033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:06.619005 18033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:41:06.619056 18033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:06.628365 18033 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:36995
I20260809 04:41:06.628391 18174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:36995 every 8 connection(s)
I20260809 04:41:06.629361 18033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:41:06.635154 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 18033
I20260809 04:41:06.635251 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:41:06.635563 6011 external_mini_cluster.cc:1428] Setting key 235d29651404d3b0322cfd6791644de7
Aug 09 04:41:06 dist-test-slave-vxj2 krb5kdc[17763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250466, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:41:06.643342 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.631209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:38453 (local address 127.5.222.254:39901)
0809 04:41:06.631357 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:06.631360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:06.632091 (+ 731us) server_negotiation.cc:407] Connection header received
0809 04:41:06.633537 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:06.633541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:06.633602 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:06.633680 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:06.634827 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.635462 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.636830 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.637055 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.637113 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.639731 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:06.639740 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:06.639742 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:41:06.639764 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:06.641320 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:06.641815 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:06.641819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:06.641821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:06.641902 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:06.642235 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:06.642239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:06.642240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:06.642454 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:06.642575 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:06.643042 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 04:41:06.643168 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:41:06.644130 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.631502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:39901 (local address 127.5.222.194:38453)
0809 04:41:06.631941 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:06.631985 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:06.633335 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:06.633842 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:06.633848 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:06.634191 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:06.634657 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.634671 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.635591 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.635600 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.636675 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.636687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.637330 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.637332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.637468 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.638090 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:06.638105 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:06.639562 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:41:06.641433 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:41:06.641439 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:41:06.641453 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:41:06.641706 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:41:06.641997 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:41:06.642001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:41:06.642003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:41:06.642137 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:41:06.642564 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:41:06.642571 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:41:06.642815 (+ 244us) client_negotiation.cc:769] Sending connection context
0809 04:41:06.642999 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 04:41:06.643321 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
I20260809 04:41:06.645382 18175 heartbeater.cc:347] Connected to a master server at 127.5.222.254:39901
I20260809 04:41:06.645653 18175 heartbeater.cc:464] Registering TS with master...
I20260809 04:41:06.646185 18175 heartbeater.cc:511] Master 127.5.222.254:39901 requested a full tablet report, sending...
I20260809 04:41:06.647284 17817 ts_manager.cc:194] Registered new tserver with Master: 1486b9470cf742c5915a708c1efa06bf (127.5.222.194:36995)
I20260809 04:41:06.647958 17817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:38453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:41:06.664990 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:39901
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:41:06.778083 18181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:06.778389 18181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:06.778477 18181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:06.782490 18181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:06.782541 18181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:06.782632 18181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:41:06.787921 18181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:39901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:06.788945 18181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:06.790016 18181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:06.799543 18186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:06.799544 18187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:06.799546 18189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:06.799926 18181 server_base.cc:1034] running on GCE node
I20260809 04:41:06.800297 18181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:06.800928 18181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:06.802088 18181 hybrid_clock.cc:648] HybridClock initialized: now 1786250466802064 us; error 51 us; skew 500 ppm
Aug 09 04:41:06 dist-test-slave-vxj2 krb5kdc[17763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250466, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:41:06.804759 18181 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:41:06.805706 18181 webserver.cc:460] Webserver started at http://127.5.222.195:45969/ using document root <none> and password file <none>
I20260809 04:41:06.806264 18181 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:06.806352 18181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:06.806600 18181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:41:06.808570 18181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "916abd1b650b4eb09abc649d0b531e1f"
format_stamp: "Formatted at 2026-08-09 04:41:06 on dist-test-slave-vxj2"
server_key: "4bd00e61f25efaabb95420df7df6843b"
server_key_iv: "52d807e53f7b2b0d85329947130fc508"
server_key_version: "encryptionkey@0"
I20260809 04:41:06.809135 18181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "916abd1b650b4eb09abc649d0b531e1f"
format_stamp: "Formatted at 2026-08-09 04:41:06 on dist-test-slave-vxj2"
server_key: "4bd00e61f25efaabb95420df7df6843b"
server_key_iv: "52d807e53f7b2b0d85329947130fc508"
server_key_version: "encryptionkey@0"
I20260809 04:41:06.812798 18181 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:41:06.815128 18197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:06.816303 18181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:41:06.816442 18181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "916abd1b650b4eb09abc649d0b531e1f"
format_stamp: "Formatted at 2026-08-09 04:41:06 on dist-test-slave-vxj2"
server_key: "4bd00e61f25efaabb95420df7df6843b"
server_key_iv: "52d807e53f7b2b0d85329947130fc508"
server_key_version: "encryptionkey@0"
I20260809 04:41:06.816550 18181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:06.826027 18181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:06.829027 18181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:06.829298 18181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:06.829869 18181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:41:06.830677 18181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:41:06.830747 18181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:06.830818 18181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:41:06.830869 18181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:06.840152 18181 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:44073
I20260809 04:41:06.840206 18323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:44073 every 8 connection(s)
I20260809 04:41:06.841228 18181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:41:06.842067 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 18181
I20260809 04:41:06.842211 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:41:06.842515 6011 external_mini_cluster.cc:1428] Setting key 61fa244bd874d081937e0af557dcae11
Aug 09 04:41:06 dist-test-slave-vxj2 krb5kdc[17763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250466, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:41:06.854494 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.843130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:37411 (local address 127.5.222.254:39901)
0809 04:41:06.843311 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:06.843316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:06.844309 (+ 993us) server_negotiation.cc:407] Connection header received
0809 04:41:06.845790 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:06.845792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:06.845838 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:06.845906 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:06.846980 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.847502 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.848555 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.848758 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.848830 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.851146 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:06.851149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:06.851151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:41:06.851173 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:06.852526 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:06.853019 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:06.853022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:06.853023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:06.853101 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:06.853503 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:06.853505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:06.853507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:06.853676 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:06.853755 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:06.854264 (+ 509us) server_negotiation.cc:299] Negotiation successful
0809 04:41:06.854362 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:41:06.855458 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.843600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:39901 (local address 127.5.222.195:37411)
0809 04:41:06.844142 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:06.844194 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:06.845606 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:06.846033 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:06.846039 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:06.846391 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:06.846864 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.846877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.847601 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.847604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.848455 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.848464 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.848875 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.848877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.849014 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.849540 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:06.849555 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:06.850994 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:41:06.852657 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:41:06.852662 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:41:06.852672 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:41:06.852929 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:41:06.853266 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:41:06.853269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:41:06.853270 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:41:06.853414 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:41:06.853807 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:41:06.853812 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:41:06.854051 (+ 239us) client_negotiation.cc:769] Sending connection context
0809 04:41:06.854220 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 04:41:06.854487 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":163,"threads_started":1}
I20260809 04:41:06.856600 18324 heartbeater.cc:347] Connected to a master server at 127.5.222.254:39901
I20260809 04:41:06.856837 18324 heartbeater.cc:464] Registering TS with master...
I20260809 04:41:06.857359 18324 heartbeater.cc:511] Master 127.5.222.254:39901 requested a full tablet report, sending...
I20260809 04:41:06.858359 17817 ts_manager.cc:194] Registered new tserver with Master: 916abd1b650b4eb09abc649d0b531e1f (127.5.222.195:44073)
I20260809 04:41:06.858938 17817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:37411
I20260809 04:41:06.868186 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:41:06.877082 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.870127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51962 (local address 127.5.222.254:39901)
0809 04:41:06.870249 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:06.870253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:06.870447 (+ 194us) server_negotiation.cc:407] Connection header received
0809 04:41:06.870548 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:06.870551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:06.870590 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:06.870661 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:06.871385 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.871884 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.872697 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.872903 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.872977 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.874432 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:06.874435 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:06.874437 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:41:06.874456 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:06.875667 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:06.876041 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:06.876043 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:06.876045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:06.876114 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:06.876338 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:06.876340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:06.876341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:06.876550 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:06.876648 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:06.876879 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 04:41:06.876968 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:41:06.879796 17817 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:41:06.883078 17817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:41:06.898387 18336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.891851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:44073 (local address 127.0.0.1:39378)
0809 04:41:06.892401 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:06.892457 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:06.892587 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:06.893868 (+ 1281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:06.893874 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:06.893911 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:41:06.894141 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.894149 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.895628 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.895631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.896803 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.896813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.897822 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.897826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.897942 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.897991 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 04:41:06.898070 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 04:41:06.898197 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":141,"spinlock_wait_cycles":4224,"thread_start_us":104,"threads_started":1}
I20260809 04:41:06.898394 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.892827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.5.222.194:36995)
0809 04:41:06.893294 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:06.893309 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:06.893337 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:41:06.893433 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:06.893439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:06.893686 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:06.893880 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:41:06.894246 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.895169 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.896874 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.897668 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.897797 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.897955 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:06.898147 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 04:41:06.898246 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":146,"threads_started":1}
I20260809 04:41:06.898885 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.891863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:36995 (local address 127.0.0.1:42760)
0809 04:41:06.892844 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:06.892856 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:06.892953 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:06.893867 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:06.893880 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:06.893912 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:41:06.894126 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.894133 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.895291 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.895294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.896740 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.896751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.897799 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.897803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.897938 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.897982 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 04:41:06.898117 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 04:41:06.898197 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"mutex_wait_us":65,"spinlock_wait_cycles":12544,"thread_start_us":114,"threads_started":1}
I20260809 04:41:06.899186 18340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.892800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39378 (local address 127.5.222.195:44073)
0809 04:41:06.893369 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:06.893382 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:06.893406 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:41:06.893503 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:06.893509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:06.893722 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:06.893878 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:41:06.894286 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.895474 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.896938 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.897682 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.898300 (+ 618us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.898463 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:06.898748 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 04:41:06.898851 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":161,"threads_started":1}
I20260809 04:41:06.899228 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.892831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42904 (local address 127.5.222.193:33607)
0809 04:41:06.893367 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:06.893382 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:06.894342 (+ 960us) server_negotiation.cc:407] Connection header received
0809 04:41:06.894420 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:06.894425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:06.894595 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:06.894742 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:41:06.894998 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.895999 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.897431 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.898259 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.898355 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.898464 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:06.898855 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 04:41:06.899080 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":82,"threads_started":1}
I20260809 04:41:06.899953 18337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.892177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:33607 (local address 127.0.0.1:42904)
0809 04:41:06.894146 (+ 1969us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:06.894160 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:06.894248 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:06.894752 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:06.894755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:06.894774 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:41:06.894862 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.894867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.896141 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.896145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.897293 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.897304 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.898529 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.898533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.898623 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.898720 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 04:41:06.898806 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:41:06.898869 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1909,"mutex_wait_us":5,"thread_start_us":103,"threads_started":1}
I20260809 04:41:06.902103 18249 tablet_service.cc:1467] Processing CreateTablet for tablet 19c2af26a4de45e5be84b864a86c8820 (DEFAULT_TABLE table=test-table [id=bc4d996d437c4fd8b122cb7f501105f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:41:06.902104 18102 tablet_service.cc:1467] Processing CreateTablet for tablet 19c2af26a4de45e5be84b864a86c8820 (DEFAULT_TABLE table=test-table [id=bc4d996d437c4fd8b122cb7f501105f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:41:06.902292 17953 tablet_service.cc:1467] Processing CreateTablet for tablet 19c2af26a4de45e5be84b864a86c8820 (DEFAULT_TABLE table=test-table [id=bc4d996d437c4fd8b122cb7f501105f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:41:06.903224 17953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19c2af26a4de45e5be84b864a86c8820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:41:06.903229 18102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19c2af26a4de45e5be84b864a86c8820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:41:06.903234 18249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19c2af26a4de45e5be84b864a86c8820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:41:06.909577 18342 tablet_bootstrap.cc:492] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67: Bootstrap starting.
I20260809 04:41:06.910574 18343 tablet_bootstrap.cc:492] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f: Bootstrap starting.
I20260809 04:41:06.910575 18344 tablet_bootstrap.cc:492] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf: Bootstrap starting.
I20260809 04:41:06.912539 18342 tablet_bootstrap.cc:654] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67: Neither blocks nor log segments found. Creating new log.
I20260809 04:41:06.912709 18343 tablet_bootstrap.cc:654] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f: Neither blocks nor log segments found. Creating new log.
I20260809 04:41:06.913261 18344 tablet_bootstrap.cc:654] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf: Neither blocks nor log segments found. Creating new log.
I20260809 04:41:06.913918 18342 log.cc:824] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67: Log is configured to *not* fsync() on all Append() calls
I20260809 04:41:06.914047 18343 log.cc:824] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:41:06.914047 18344 log.cc:824] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf: Log is configured to *not* fsync() on all Append() calls
I20260809 04:41:06.915760 18342 tablet_bootstrap.cc:492] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67: No bootstrap required, opened a new log
I20260809 04:41:06.915767 18343 tablet_bootstrap.cc:492] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f: No bootstrap required, opened a new log
I20260809 04:41:06.915967 18343 ts_tablet_manager.cc:1397] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 04:41:06.915968 18342 ts_tablet_manager.cc:1397] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260809 04:41:06.917052 18344 tablet_bootstrap.cc:492] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf: No bootstrap required, opened a new log
I20260809 04:41:06.917251 18344 ts_tablet_manager.cc:1397] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 04:41:06.919135 18343 raft_consensus.cc:348] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916abd1b650b4eb09abc649d0b531e1f" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44073 } } peers { permanent_uuid: "1486b9470cf742c5915a708c1efa06bf" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36995 } } peers { permanent_uuid: "4c22a8821d284adc909c9b1c9ef19e67" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 33607 } }
I20260809 04:41:06.919374 18343 raft_consensus.cc:374] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:41:06.919438 18343 raft_consensus.cc:729] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 916abd1b650b4eb09abc649d0b531e1f, State: Initialized, Role: FOLLOWER
I20260809 04:41:06.919943 18343 consensus_queue.cc:260] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916abd1b650b4eb09abc649d0b531e1f" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44073 } } peers { permanent_uuid: "1486b9470cf742c5915a708c1efa06bf" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36995 } } peers { permanent_uuid: "4c22a8821d284adc909c9b1c9ef19e67" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 33607 } }
I20260809 04:41:06.919847 18342 raft_consensus.cc:348] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916abd1b650b4eb09abc649d0b531e1f" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44073 } } peers { permanent_uuid: "1486b9470cf742c5915a708c1efa06bf" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36995 } } peers { permanent_uuid: "4c22a8821d284adc909c9b1c9ef19e67" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 33607 } }
I20260809 04:41:06.920094 18342 raft_consensus.cc:374] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:41:06.920150 18342 raft_consensus.cc:729] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c22a8821d284adc909c9b1c9ef19e67, State: Initialized, Role: FOLLOWER
I20260809 04:41:06.920583 18324 heartbeater.cc:503] Master 127.5.222.254:39901 was elected leader, sending a full tablet report...
I20260809 04:41:06.920634 18342 consensus_queue.cc:260] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916abd1b650b4eb09abc649d0b531e1f" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44073 } } peers { permanent_uuid: "1486b9470cf742c5915a708c1efa06bf" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36995 } } peers { permanent_uuid: "4c22a8821d284adc909c9b1c9ef19e67" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 33607 } }
I20260809 04:41:06.920945 18343 ts_tablet_manager.cc:1428] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 04:41:06.920985 18344 raft_consensus.cc:348] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916abd1b650b4eb09abc649d0b531e1f" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44073 } } peers { permanent_uuid: "1486b9470cf742c5915a708c1efa06bf" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36995 } } peers { permanent_uuid: "4c22a8821d284adc909c9b1c9ef19e67" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 33607 } }
I20260809 04:41:06.921237 18344 raft_consensus.cc:374] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:41:06.921305 18344 raft_consensus.cc:729] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1486b9470cf742c5915a708c1efa06bf, State: Initialized, Role: FOLLOWER
I20260809 04:41:06.921613 18342 ts_tablet_manager.cc:1428] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260809 04:41:06.921700 18028 heartbeater.cc:503] Master 127.5.222.254:39901 was elected leader, sending a full tablet report...
I20260809 04:41:06.921782 18344 consensus_queue.cc:260] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916abd1b650b4eb09abc649d0b531e1f" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44073 } } peers { permanent_uuid: "1486b9470cf742c5915a708c1efa06bf" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36995 } } peers { permanent_uuid: "4c22a8821d284adc909c9b1c9ef19e67" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 33607 } }
I20260809 04:41:06.922684 18344 ts_tablet_manager.cc:1428] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 04:41:06.923000 18175 heartbeater.cc:503] Master 127.5.222.254:39901 was elected leader, sending a full tablet report...
I20260809 04:41:06.932785 18349 raft_consensus.cc:482] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:41:06.933002 18349 raft_consensus.cc:504] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916abd1b650b4eb09abc649d0b531e1f" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44073 } } peers { permanent_uuid: "1486b9470cf742c5915a708c1efa06bf" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36995 } } peers { permanent_uuid: "4c22a8821d284adc909c9b1c9ef19e67" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 33607 } }
I20260809 04:41:06.934248 18349 leader_election.cc:288] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 916abd1b650b4eb09abc649d0b531e1f (127.5.222.195:44073), 1486b9470cf742c5915a708c1efa06bf (127.5.222.194:36995)
I20260809 04:41:06.939935 18340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.934548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:49393 (local address 127.5.222.195:44073)
0809 04:41:06.934689 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:06.934694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:06.934840 (+ 146us) server_negotiation.cc:407] Connection header received
0809 04:41:06.935106 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:06.935109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:06.935161 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:06.935256 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:41:06.935675 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.936462 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.938474 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.939117 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.939229 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.939280 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:06.939720 (+ 440us) server_negotiation.cc:299] Negotiation successful
0809 04:41:06.939836 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:41:06.939929 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.934771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:57359 (local address 127.5.222.194:36995)
0809 04:41:06.934891 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:06.934897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:06.935069 (+ 172us) server_negotiation.cc:407] Connection header received
0809 04:41:06.935126 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:06.935129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:06.935174 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:06.935257 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:41:06.935677 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.936471 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.938474 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.939094 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.939229 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.939280 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:06.939720 (+ 440us) server_negotiation.cc:299] Negotiation successful
0809 04:41:06.939794 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:41:06.940014 18351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.934427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:44073 (local address 127.5.222.193:49393)
0809 04:41:06.934723 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:06.934746 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:06.934920 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:06.935286 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:06.935289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:06.935327 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:41:06.935549 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.935557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.936629 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.936631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.938328 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.938340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.939235 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.939238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.939514 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.939604 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 04:41:06.939690 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:41:06.939829 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":302,"spinlock_wait_cycles":28288,"thread_start_us":93,"threads_started":1}
I20260809 04:41:06.940219 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.934584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:36995 (local address 127.5.222.193:57359)
0809 04:41:06.934946 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:06.934961 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:06.935052 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:06.935286 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:06.935289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:06.935329 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:41:06.935552 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.935557 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.936624 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.936628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.938328 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:06.938340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.939235 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.939238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:06.939514 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.939602 (+ 88us) client_negotiation.cc:769] Sending connection context
0809 04:41:06.939690 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:41:06.939837 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":396,"spinlock_wait_cycles":5120,"thread_start_us":161,"threads_started":1}
I20260809 04:41:06.940821 18270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c2af26a4de45e5be84b864a86c8820" candidate_uuid: "4c22a8821d284adc909c9b1c9ef19e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "916abd1b650b4eb09abc649d0b531e1f" is_pre_election: true
I20260809 04:41:06.940836 18127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c2af26a4de45e5be84b864a86c8820" candidate_uuid: "4c22a8821d284adc909c9b1c9ef19e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1486b9470cf742c5915a708c1efa06bf" is_pre_election: true
I20260809 04:41:06.941186 18127 raft_consensus.cc:2454] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c22a8821d284adc909c9b1c9ef19e67 in term 0.
I20260809 04:41:06.941190 18270 raft_consensus.cc:2454] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c22a8821d284adc909c9b1c9ef19e67 in term 0.
I20260809 04:41:06.941649 17903 leader_election.cc:302] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c22a8821d284adc909c9b1c9ef19e67, 916abd1b650b4eb09abc649d0b531e1f; no voters:
I20260809 04:41:06.941900 18349 raft_consensus.cc:2781] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:41:06.941989 18349 raft_consensus.cc:482] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:41:06.942041 18349 raft_consensus.cc:3037] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:41:06.943078 18349 raft_consensus.cc:504] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916abd1b650b4eb09abc649d0b531e1f" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44073 } } peers { permanent_uuid: "1486b9470cf742c5915a708c1efa06bf" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36995 } } peers { permanent_uuid: "4c22a8821d284adc909c9b1c9ef19e67" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 33607 } }
I20260809 04:41:06.943430 18349 leader_election.cc:288] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [CANDIDATE]: Term 1 election: Requested vote from peers 916abd1b650b4eb09abc649d0b531e1f (127.5.222.195:44073), 1486b9470cf742c5915a708c1efa06bf (127.5.222.194:36995)
I20260809 04:41:06.943737 18270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c2af26a4de45e5be84b864a86c8820" candidate_uuid: "4c22a8821d284adc909c9b1c9ef19e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "916abd1b650b4eb09abc649d0b531e1f"
I20260809 04:41:06.943876 18270 raft_consensus.cc:3037] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:41:06.944029 18127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c2af26a4de45e5be84b864a86c8820" candidate_uuid: "4c22a8821d284adc909c9b1c9ef19e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1486b9470cf742c5915a708c1efa06bf"
I20260809 04:41:06.944147 18127 raft_consensus.cc:3037] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:41:06.945013 18270 raft_consensus.cc:2454] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c22a8821d284adc909c9b1c9ef19e67 in term 1.
I20260809 04:41:06.945269 18127 raft_consensus.cc:2454] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c22a8821d284adc909c9b1c9ef19e67 in term 1.
I20260809 04:41:06.945297 17903 leader_election.cc:302] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c22a8821d284adc909c9b1c9ef19e67, 916abd1b650b4eb09abc649d0b531e1f; no voters:
I20260809 04:41:06.945508 18349 raft_consensus.cc:2781] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:41:06.945771 18349 raft_consensus.cc:686] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [term 1 LEADER]: Becoming Leader. State: Replica: 4c22a8821d284adc909c9b1c9ef19e67, State: Running, Role: LEADER
I20260809 04:41:06.946128 18349 consensus_queue.cc:237] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916abd1b650b4eb09abc649d0b531e1f" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44073 } } peers { permanent_uuid: "1486b9470cf742c5915a708c1efa06bf" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36995 } } peers { permanent_uuid: "4c22a8821d284adc909c9b1c9ef19e67" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 33607 } }
I20260809 04:41:06.949362 17816 catalog_manager.cc:5495] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c22a8821d284adc909c9b1c9ef19e67 (127.5.222.193). New cstate: current_term: 1 leader_uuid: "4c22a8821d284adc909c9b1c9ef19e67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "916abd1b650b4eb09abc649d0b531e1f" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1486b9470cf742c5915a708c1efa06bf" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c22a8821d284adc909c9b1c9ef19e67" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 33607 } health_report { overall_health: HEALTHY } } }
I20260809 04:41:06.964725 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:06.961235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.5.222.193:33607)
0809 04:41:06.961387 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:06.961392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:06.961406 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:41:06.961508 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:06.961510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:06.961559 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:06.961675 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:41:06.961945 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.962445 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.963330 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:06.963553 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:06.963665 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:06.963966 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:41:06.964329 (+ 363us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:41:06.964476 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:06.964536 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 04:41:06.964590 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:41:06.970543 18127 raft_consensus.cc:1261] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf [term 1 FOLLOWER]: Refusing update from remote peer 4c22a8821d284adc909c9b1c9ef19e67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:41:06.970628 18270 raft_consensus.cc:1261] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f [term 1 FOLLOWER]: Refusing update from remote peer 4c22a8821d284adc909c9b1c9ef19e67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:41:06.971264 18355 consensus_queue.cc:1035] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1486b9470cf742c5915a708c1efa06bf" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:41:06.971479 18349 consensus_queue.cc:1035] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "916abd1b650b4eb09abc649d0b531e1f" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 44073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:41:06.978916 18358 mvcc.cc:205] Tried to move back new op lower bound from 7316481912704303104 to 7316481912613998592. Current Snapshot: MvccSnapshot[applied={T|T < 7316481912704303104}]
I20260809 04:41:06.980294 18359 mvcc.cc:205] Tried to move back new op lower bound from 7316481912704303104 to 7316481912613998592. Current Snapshot: MvccSnapshot[applied={T|T < 7316481912704303104}]
I20260809 04:41:06.988385 17816 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:41:06.988605 17816 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:41:06.991092 17816 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e4ab1c226ff467f820f22dcafaf963e: Sending DeleteTablet for 3 replicas of tablet 19c2af26a4de45e5be84b864a86c8820
I20260809 04:41:06.991977 18249 tablet_service.cc:1514] Processing DeleteTablet for tablet 19c2af26a4de45e5be84b864a86c8820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:41:06 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:39378
I20260809 04:41:06.992197 17953 tablet_service.cc:1514] Processing DeleteTablet for tablet 19c2af26a4de45e5be84b864a86c8820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:41:06 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:42904
I20260809 04:41:06.992519 18102 tablet_service.cc:1514] Processing DeleteTablet for tablet 19c2af26a4de45e5be84b864a86c8820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:41:06 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:42760
I20260809 04:41:06.992749 18371 tablet_replica.cc:330] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67: stopping tablet replica
I20260809 04:41:06.993105 18371 raft_consensus.cc:2229] T 19c2af26a4de45e5be84b864a86c8820 P 4c22a8821d284adc909c9b1c9ef19e67 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:41:06.993391 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 17884
I20260809 04:41:06.993275 18370 tablet_replica.cc:330] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f: stopping tablet replica
I20260809 04:41:06.996199 18373 tablet_replica.cc:330] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf: stopping tablet replica
I20260809 04:41:06.996472 18370 raft_consensus.cc:2229] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:41:06.996630 18373 raft_consensus.cc:2229] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:41:06.996819 18370 raft_consensus.cc:2258] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:41:06.996981 18373 raft_consensus.cc:2258] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:41:06.998368 18373 ts_tablet_manager.cc:1905] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:41:07.000563 18373 ts_tablet_manager.cc:1918] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:41:07.000646 18373 log.cc:1196] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/19c2af26a4de45e5be84b864a86c8820
I20260809 04:41:07.001003 18373 ts_tablet_manager.cc:1939] T 19c2af26a4de45e5be84b864a86c8820 P 1486b9470cf742c5915a708c1efa06bf: Deleting consensus metadata
W20260809 04:41:07.001667 17797 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:33607 (error 108)
I20260809 04:41:07.002166 18370 ts_tablet_manager.cc:1905] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:41:07.002364 17795 catalog_manager.cc:4841] TS 1486b9470cf742c5915a708c1efa06bf (127.5.222.194:36995): tablet 19c2af26a4de45e5be84b864a86c8820 (table test-table [id=bc4d996d437c4fd8b122cb7f501105f6]) successfully deleted
I20260809 04:41:07.002843 18337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:07.002542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:33607 (local address 127.0.0.1:42914)
0809 04:41:07.002658 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:07.002739 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:33607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 04:41:07.002995 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 18033
W20260809 04:41:07.003072 17797 catalog_manager.cc:4584] TS 4c22a8821d284adc909c9b1c9ef19e67 (127.5.222.193:33607): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19c2af26a4de45e5be84b864a86c8820: Network error: Client connection negotiation failed: client connection to 127.5.222.193:33607: connect: Connection refused (error 111)
I20260809 04:41:07.006330 18370 ts_tablet_manager.cc:1918] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:41:07.006462 18370 log.cc:1196] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/19c2af26a4de45e5be84b864a86c8820
I20260809 04:41:07.006786 18370 ts_tablet_manager.cc:1939] T 19c2af26a4de45e5be84b864a86c8820 P 916abd1b650b4eb09abc649d0b531e1f: Deleting consensus metadata
I20260809 04:41:07.008330 17796 catalog_manager.cc:4841] TS 916abd1b650b4eb09abc649d0b531e1f (127.5.222.195:44073): tablet 19c2af26a4de45e5be84b864a86c8820 (table test-table [id=bc4d996d437c4fd8b122cb7f501105f6]) successfully deleted
I20260809 04:41:07.012691 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 18181
I20260809 04:41:07.019058 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 17777
2026-08-09T04:41:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2338 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:41:07 dist-test-slave-vxj2 krb5kdc[18377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:41:07 dist-test-slave-vxj2 krb5kdc[18377](info): set up 2 sockets
Aug 09 04:41:07 dist-test-slave-vxj2 krb5kdc[18377](info): commencing operation
krb5kdc: starting...
W20260809 04:41:08.117791 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:41:08 dist-test-slave-vxj2 krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:41:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:41:08Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.222.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.222.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/rangeradmin_127.5.222.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/rangeradmin_127.5.222.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.222.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.222.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/rangerlookup_127.5.222.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/rangerlookup_127.5.222.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.222.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.222.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/HTTP_127.5.222.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/HTTP_127.5.222.212@KRBTEST.COM.keytab.
I20260809 04:41:08.291604 6011 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskxJEoz3/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/postgres -l logfile start
2026-08-09 04:41:10.877 UTC [18409] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 04:41:10.877 UTC [18409] LOG: listening on IPv4 address "127.5.222.212", port 60893
2026-08-09 04:41:10.881 UTC [18409] LOG: listening on Unix socket "/tmp/.s.PGSQL.60893"
2026-08-09 04:41:10.903 UTC [18413] LOG: database system was shut down at 2026-08-09 04:41:09 UTC
2026-08-09 04:41:10.907 UTC [18409] LOG: database system is ready to accept connections
I20260809 04:41:11.917073 6011 mini_postgres.cc:96] Postgres bound to 60893
2026-08-09 04:41:11.924 UTC [18421] FATAL: database "slave" does not exist
127.5.222.212:60893 - accepting connections
I20260809 04:41:11.925320 6011 mini_ranger.cc:161] Starting Ranger...
I20260809 04:41:11.941196 6011 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 04:41:12.450037 6011 mini_ranger.cc:87] Created ranger Postgres database
I20260809 04:41:12.450160 6011 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-admin
2026-08-09 04:41:12,775 [I] DB FLAVOR :POSTGRES
2026-08-09 04:41:12,775 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:41:12,775 [I] Checking connection..
2026-08-09 04:41:12,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 04:41:13,029 [I] Checking connection passed.
2026-08-09 04:41:13,030 [I] --------- Verifying version history table ---------
2026-08-09 04:41:13,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:41:13,268 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:41:13,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:41:13,527 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:41:13,527 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 04:41:13,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 04:41:13.756 UTC [18495] WARNING: there is no transaction in progress
2026-08-09 04:41:13,764 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 04:41:13,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:41:14,012 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 04:41:14,013 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:41:14,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 04:41:14,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 04:41:14,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vxj2.c.gcp-upstream.internal','N') ;"
2026-08-09 04:41:14,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:41:14,954 [I] Table x_portal_user does not exist in database ranger
2026-08-09 04:41:14,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:41:15,212 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 04:41:15,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:41:15,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:41:15,717 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 04:41:15,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 04:41:16.718 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.737 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.756 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.773 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.791 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.812 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.872 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.881 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.896 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.907 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.918 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.933 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.943 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.949 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.956 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:16.964 UTC [18614] WARNING: there is no transaction in progress
2026-08-09 04:41:17,623 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 04:41:17,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 04:41:17,855 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 04:41:17,856 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:41:18,117 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 04:41:18,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:41:18,385 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 04:41:18,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:41:18,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:41:18,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vxj2.c.gcp-upstream.internal';"
2026-08-09 04:41:19,096 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 04:41:19,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 04:41:19,358 [I] DB_PATCHES have already been applied
I20260809 04:41:19.363900 6011 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-admin:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskxJEoz3/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 04:41:19.363967 6011 mini_ranger.cc:193] Using host: 127.5.222.212
I20260809 04:41:19.368089 6011 mini_ranger.cc:227] Ranger admin URL: http://127.5.222.212:58837
Aug 09, 2026 4:41:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:41:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:41:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:41:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:41:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:41:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:41:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 04:41:20.556089 6011 mini_ranger.cc:160] Time spent starting Ranger: real 8.631s user 0.000s sys 0.012s
Aug 09, 2026 4:41:40 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:41:40 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 4:41:40 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:41:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:41:40 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:41:41 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 04:41:41.741520 6011 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.222.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.222.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/rangerkms_127.5.222.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/rangerkms_127.5.222.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.222.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.222.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.222.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/HTTP_127.5.222.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.222.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/HTTP_127.5.222.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 04:41:41 dist-test-slave-vxj2 krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250501, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 04:41:41.831605 6011 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 04:41:41.841776 6011 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 04:41:42.726879 6011 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 04:41:42.726994 6011 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-kms
I20260809 04:41:42.727105 6011 mini_ranger_kms.cc:227] Using postgres at 127.5.222.212:60893
2026-08-09 04:41:43,082 [I] DB FLAVOR :POSTGRES
2026-08-09 04:41:43,082 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:41:43,082 [I] Checking connection
2026-08-09 04:41:43,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 04:41:43,361 [I] connection success
2026-08-09 04:41:43,361 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 04:41:43,361 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 04:41:43,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 04:41:43,610 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 04:41:43,611 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:41:43,611 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 04:41:43,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:60893/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 04:41:43,866 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 04:41:44.325882 6011 mini_ranger_kms.cc:310] Created kms service
I20260809 04:41:44.526917 6011 mini_ranger_kms.cc:326] Created kudu user
I20260809 04:41:44.573297 6011 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 04:41:44 dist-test-slave-vxj2 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250501, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:41:44.998323 6011 mini_ranger_kms.cc:384] Added ranger policy
I20260809 04:41:44.998495 6011 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-kms:/tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 04:41:44.998517 6011 mini_ranger_kms.cc:240] Using host: 127.5.222.212
I20260809 04:41:45.002944 6011 mini_ranger_kms.cc:276] Ranger KMS PID: 18811
I20260809 04:41:45.003089 6011 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.222.212:35973
Aug 09, 2026 4:41:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:41:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:41:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:41:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:41:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:41:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 04:41:47.416150 6011 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.585s user 0.008s sys 0.328s
I20260809 04:41:47.416289 6011 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 04:41:47.416421 6011 mini_ranger_kms.cc:401] 127.5.222.212:35973/kms/v1/keys
Aug 09 04:41:47 dist-test-slave-vxj2 krb5kdc[18377](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250507, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.222.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 04:41:48 dist-test-slave-vxj2 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250507, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.222.212@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
Aug 09 04:41:48 dist-test-slave-vxj2 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250507, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.222.212@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
Aug 09 04:41:50 dist-test-slave-vxj2 krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:41:50.860765 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:35703
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:35703
--ranger_config_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.222.212:35973/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:41:51.009276 18869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:51.009645 18869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:51.009774 18869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:51.014783 18869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:41:51.014892 18869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:51.014941 18869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:51.015014 18869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:41:51.015065 18869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:41:51.021289 18869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.222.212:35973/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:35703
--ranger_config_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:35703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:51.022445 18869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:51.023536 18869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:51.033718 18875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:51.033672 18877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:51.033740 18874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:51.034001 18869 server_base.cc:1034] running on GCE node
I20260809 04:41:51.034754 18869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:51.036037 18869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:51.037221 18869 hybrid_clock.cc:648] HybridClock initialized: now 1786250511037201 us; error 41 us; skew 500 ppm
Aug 09 04:41:51 dist-test-slave-vxj2 krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250511, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:41:51.040509 18869 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:41:51.041469 18869 webserver.cc:460] Webserver started at http://127.5.222.254:43209/ using document root <none> and password file <none>
I20260809 04:41:51.042016 18869 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:51.042096 18869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:51.042312 18869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:41:51 dist-test-slave-vxj2 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250511, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:41:51.102099 18869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "3722aaa469944960882b12e51afbd074"
format_stamp: "Formatted at 2026-08-09 04:41:51 on dist-test-slave-vxj2"
server_key: "a7231457dcb8d6a8bd70078842b54c42"
server_key_iv: "c05104cebea993ef80a80515b0fa0d56"
server_key_version: "kuduclusterkey@0"
I20260809 04:41:51.103051 18869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "3722aaa469944960882b12e51afbd074"
format_stamp: "Formatted at 2026-08-09 04:41:51 on dist-test-slave-vxj2"
server_key: "a7231457dcb8d6a8bd70078842b54c42"
server_key_iv: "c05104cebea993ef80a80515b0fa0d56"
server_key_version: "kuduclusterkey@0"
I20260809 04:41:51.108177 18869 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:41:51.125831 18884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:51.127350 18869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:41:51.127530 18869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "3722aaa469944960882b12e51afbd074"
format_stamp: "Formatted at 2026-08-09 04:41:51 on dist-test-slave-vxj2"
server_key: "a7231457dcb8d6a8bd70078842b54c42"
server_key_iv: "c05104cebea993ef80a80515b0fa0d56"
server_key_version: "kuduclusterkey@0"
I20260809 04:41:51.127669 18869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:51.137632 18869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:51.140678 18869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:51.140964 18869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:51.153702 18869 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:35703
I20260809 04:41:51.153746 18941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:35703 every 8 connection(s)
I20260809 04:41:51.154814 18869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:41:51.157888 18942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:41:51.160724 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 18869
I20260809 04:41:51.160868 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:41:51.161195 6011 external_mini_cluster.cc:1428] Setting key 8d093e7df692fc82975a2da2689f6668
I20260809 04:41:51.166056 18942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074: Bootstrap starting.
I20260809 04:41:51.169557 18942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074: Neither blocks nor log segments found. Creating new log.
I20260809 04:41:51.170934 18942 log.cc:824] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:41:51 dist-test-slave-vxj2 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:41:51.173765 18942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074: No bootstrap required, opened a new log
I20260809 04:41:51.178884 18942 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3722aaa469944960882b12e51afbd074" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35703 } }
I20260809 04:41:51.179140 18942 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:41:51.179241 18942 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3722aaa469944960882b12e51afbd074, State: Initialized, Role: FOLLOWER
I20260809 04:41:51.179550 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:51.162712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56346 (local address 127.5.222.254:35703)
0809 04:41:51.163220 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:51.163266 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:51.163312 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:41:51.164845 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:51.164873 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:51.165305 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:51.166397 (+ 1092us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:51.166944 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:51.168447 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:51.169851 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:51.170520 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:51.170673 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:51.173974 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:51.173980 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:51.173997 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:41:51.174027 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:51.176108 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:51.176712 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:51.176717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:51.176723 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:51.176855 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:51.177231 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:51.177234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:51.177235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:51.177736 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:51.177956 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:51.178338 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 04:41:51.178613 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":141,"threads_started":1}
I20260809 04:41:51.179780 18942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3722aaa469944960882b12e51afbd074" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35703 } }
I20260809 04:41:51.179949 18942 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:41:51.180022 18942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:41:51.180127 18942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:41:51.181460 18942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3722aaa469944960882b12e51afbd074" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35703 } }
I20260809 04:41:51.181830 18942 leader_election.cc:302] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3722aaa469944960882b12e51afbd074; no voters:
I20260809 04:41:51.182219 18942 leader_election.cc:288] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:41:51.182327 18949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:41:51.182607 18949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [term 1 LEADER]: Becoming Leader. State: Replica: 3722aaa469944960882b12e51afbd074, State: Running, Role: LEADER
I20260809 04:41:51.182986 18949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3722aaa469944960882b12e51afbd074" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35703 } }
I20260809 04:41:51.183568 18942 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:41:51.184212 18949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3722aaa469944960882b12e51afbd074. Latest consensus state: current_term: 1 leader_uuid: "3722aaa469944960882b12e51afbd074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3722aaa469944960882b12e51afbd074" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35703 } } }
I20260809 04:41:51.184332 18949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [sys.catalog]: This master's current role is: LEADER
I20260809 04:41:51.184787 18950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3722aaa469944960882b12e51afbd074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3722aaa469944960882b12e51afbd074" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35703 } } }
I20260809 04:41:51.184940 18950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074 [sys.catalog]: This master's current role is: LEADER
I20260809 04:41:51.186519 18942 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 04:41:51.187963 18956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:41:51.195715 18956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:41:51.204468 18956 catalog_manager.cc:1347] Generated new cluster ID: 62b09abe0d6749ddb79f1d2ab958233a
I20260809 04:41:51.204541 18956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:41:51.246202 18956 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:41:51.247845 18956 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:41:51.253826 18956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074: Generated new TSK 0
I20260809 04:41:51.254878 18956 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:41:52.100566 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.222.212:35973/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35703
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:41:52.244305 18991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:52.244593 18991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:52.244669 18991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:52.248883 18991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:52.248960 18991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:52.249045 18991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:41:52.254717 18991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.222.212:35973/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:52.256012 18991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:52.257051 18991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:52.268013 18997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:52.268013 18998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:52.268764 19000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:52.269393 18991 server_base.cc:1034] running on GCE node
I20260809 04:41:52.269908 18991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:52.270610 18991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:52.271816 18991 hybrid_clock.cc:648] HybridClock initialized: now 1786250512271744 us; error 83 us; skew 500 ppm
Aug 09 04:41:52 dist-test-slave-vxj2 krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250512, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:41:52.275621 18991 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:41:52.276834 18991 webserver.cc:460] Webserver started at http://127.5.222.193:37713/ using document root <none> and password file <none>
I20260809 04:41:52.277642 18991 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:52.277747 18991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:52.278021 18991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:41:52 dist-test-slave-vxj2 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250512, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:41:52.293615 18991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "6bc356e377a94e67839632bd52f1ca38"
format_stamp: "Formatted at 2026-08-09 04:41:52 on dist-test-slave-vxj2"
server_key: "716e9c87d3852e5742b63e768fb8b8b0"
server_key_iv: "7b4c837b4bb0f70e298ae32a3e25f254"
server_key_version: "kuduclusterkey@0"
I20260809 04:41:52.294318 18991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "6bc356e377a94e67839632bd52f1ca38"
format_stamp: "Formatted at 2026-08-09 04:41:52 on dist-test-slave-vxj2"
server_key: "716e9c87d3852e5742b63e768fb8b8b0"
server_key_iv: "7b4c837b4bb0f70e298ae32a3e25f254"
server_key_version: "kuduclusterkey@0"
I20260809 04:41:52.299922 18991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:41:52.323350 19010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:52.325258 18991 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.004s sys 0.001s
I20260809 04:41:52.325399 18991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "6bc356e377a94e67839632bd52f1ca38"
format_stamp: "Formatted at 2026-08-09 04:41:52 on dist-test-slave-vxj2"
server_key: "716e9c87d3852e5742b63e768fb8b8b0"
server_key_iv: "7b4c837b4bb0f70e298ae32a3e25f254"
server_key_version: "kuduclusterkey@0"
I20260809 04:41:52.325546 18991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:52.344401 18991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:52.347990 18991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:52.348161 18991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:52.348958 18991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:41:52.350078 18991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:41:52.350142 18991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:52.350201 18991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:41:52.350224 18991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:52.366634 18991 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:39015
I20260809 04:41:52.367887 18991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:41:52.370692 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 18991
I20260809 04:41:52.370857 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:41:52.371282 6011 external_mini_cluster.cc:1428] Setting key 5b44b6adf9af047d689c145ca592929a
I20260809 04:41:52.369561 19138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:39015 every 8 connection(s)
Aug 09 04:41:52 dist-test-slave-vxj2 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250512, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
I20260809 04:41:52.389837 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:52.372797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:35703 (local address 127.5.222.193:50153)
0809 04:41:52.373541 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:52.373602 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:52.374980 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:52.375954 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:52.375963 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:52.376519 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:52.377296 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:52.377316 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:52.378453 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:52.378456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:52.379902 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:52.379925 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:52.380851 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:52.380855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:52.381055 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:52.381854 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:52.381874 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:52.383670 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:41:52.386232 (+ 2562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:41:52.386238 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:41:52.386254 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:41:52.386646 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:41:52.387027 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:41:52.387030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:41:52.387032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:41:52.387221 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:41:52.387741 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:41:52.387748 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:41:52.388121 (+ 373us) client_negotiation.cc:769] Sending connection context
0809 04:41:52.388336 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 04:41:52.388698 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":170,"threads_started":1}
I20260809 04:41:52.390434 19143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:52.373424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:50153 (local address 127.5.222.254:35703)
0809 04:41:52.375447 (+ 2023us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:52.375452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:52.375468 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:41:52.375507 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:52.375510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:52.375553 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:52.375636 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:52.377464 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:52.378307 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:52.380197 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:52.380532 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:52.380649 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:52.383892 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:52.383896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:52.383899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:41:52.383925 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:52.386093 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:52.386812 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:52.386816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:52.386818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:52.386915 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:52.387333 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:52.387336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:52.387338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:52.387572 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:52.387670 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:52.390116 (+ 2446us) server_negotiation.cc:299] Negotiation successful
0809 04:41:52.390268 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1944,"thread_start_us":93,"threads_started":1}
I20260809 04:41:52.392103 19139 heartbeater.cc:347] Connected to a master server at 127.5.222.254:35703
I20260809 04:41:52.392400 19139 heartbeater.cc:464] Registering TS with master...
I20260809 04:41:52.393074 19139 heartbeater.cc:511] Master 127.5.222.254:35703 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:41:52.394928 18903 ts_manager.cc:194] Registered new tserver with Master: 6bc356e377a94e67839632bd52f1ca38 (127.5.222.193:39015)
I20260809 04:41:52.396687 18903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:50153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:41:52.421880 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.222.212:35973/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35703
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:41:52 dist-test-slave-vxj2 krb5kdc[18377](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250512, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 04:41:52.620934 19149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:52.621353 19149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:52.621448 19149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:52.627651 19149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:52.627728 19149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:52.627820 19149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:41:52.635780 19149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.222.212:35973/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:52.637111 19149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:52.638417 19149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:52.651881 19156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:52.651850 19160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:52.652220 19158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:52.653589 19149 server_base.cc:1034] running on GCE node
I20260809 04:41:52.654018 19149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:52.654770 19149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:52.655972 19149 hybrid_clock.cc:648] HybridClock initialized: now 1786250512655947 us; error 53 us; skew 500 ppm
Aug 09 04:41:52 dist-test-slave-vxj2 krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250512, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:41:52.660471 19149 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:41:52.661861 19149 webserver.cc:460] Webserver started at http://127.5.222.194:41569/ using document root <none> and password file <none>
I20260809 04:41:52.662767 19149 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:52.662894 19149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:52.663172 19149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:41:52 dist-test-slave-vxj2 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250512, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:41:52.687037 19149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "c97d086ca3034755a36ea3feec06edca"
format_stamp: "Formatted at 2026-08-09 04:41:52 on dist-test-slave-vxj2"
server_key: "5bfd79ac5870bdd2e287d4127116d0be"
server_key_iv: "15d2f6373dc1ace9d1817b0175abbfc3"
server_key_version: "kuduclusterkey@0"
I20260809 04:41:52.687758 19149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "c97d086ca3034755a36ea3feec06edca"
format_stamp: "Formatted at 2026-08-09 04:41:52 on dist-test-slave-vxj2"
server_key: "5bfd79ac5870bdd2e287d4127116d0be"
server_key_iv: "15d2f6373dc1ace9d1817b0175abbfc3"
server_key_version: "kuduclusterkey@0"
I20260809 04:41:52.692948 19149 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 04:41:52.712809 19167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:52.714524 19149 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260809 04:41:52.714675 19149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "c97d086ca3034755a36ea3feec06edca"
format_stamp: "Formatted at 2026-08-09 04:41:52 on dist-test-slave-vxj2"
server_key: "5bfd79ac5870bdd2e287d4127116d0be"
server_key_iv: "15d2f6373dc1ace9d1817b0175abbfc3"
server_key_version: "kuduclusterkey@0"
I20260809 04:41:52.714823 19149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:52.731184 19149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:52.734787 19149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:52.734980 19149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:52.735621 19149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:41:52.736626 19149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:41:52.736689 19149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:52.736753 19149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:41:52.736789 19149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:52.757416 19149 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:40147
I20260809 04:41:52.758833 19149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:41:52.762723 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 19149
I20260809 04:41:52.762852 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:41:52.763156 6011 external_mini_cluster.cc:1428] Setting key 71d75386725a97f8c8adfe385b3cfa94
I20260809 04:41:52.771512 19296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:40147 every 8 connection(s)
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
Aug 09 04:41:52 dist-test-slave-vxj2 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250512, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:41:52.797122 19143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:52.776680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:51027 (local address 127.5.222.254:35703)
0809 04:41:52.776828 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:52.776833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:52.779210 (+ 2377us) server_negotiation.cc:407] Connection header received
0809 04:41:52.780970 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:52.780975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:52.781050 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:52.781114 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:52.783218 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:52.784171 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:52.786137 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:52.786540 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:52.786652 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:52.792609 (+ 5957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:52.792613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:52.792616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:41:52.792646 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:52.794586 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:52.795215 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:52.795219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:52.795220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:52.795304 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:52.795700 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:52.795703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:52.795705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:52.795935 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:52.796020 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:52.796692 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 04:41:52.796862 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
I20260809 04:41:52.798426 19300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:52.775228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:35703 (local address 127.5.222.194:51027)
0809 04:41:52.779011 (+ 3783us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:52.779078 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:52.780645 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:52.781644 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:52.781656 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:52.782161 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:52.782998 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:52.783023 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:52.784347 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:52.784351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:52.785943 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:52.785955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:52.786884 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:52.786888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:52.787180 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:52.790299 (+ 3119us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:52.790320 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:52.792353 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:41:52.794722 (+ 2369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:41:52.794728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:41:52.794744 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:41:52.795093 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:41:52.795415 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:41:52.795418 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:41:52.795420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:41:52.795588 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:41:52.796090 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:41:52.796096 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:41:52.796400 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 04:41:52.796603 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 04:41:52.796935 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3500,"thread_start_us":329,"threads_started":1}
I20260809 04:41:52.800053 19297 heartbeater.cc:347] Connected to a master server at 127.5.222.254:35703
I20260809 04:41:52.800362 19297 heartbeater.cc:464] Registering TS with master...
I20260809 04:41:52.801119 19297 heartbeater.cc:511] Master 127.5.222.254:35703 requested a full tablet report, sending...
I20260809 04:41:52.802553 18903 ts_manager.cc:194] Registered new tserver with Master: c97d086ca3034755a36ea3feec06edca (127.5.222.194:40147)
I20260809 04:41:52.803408 18903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:51027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:41:52.820977 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.222.212:35973/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35703
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:41:53.041379 19302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:41:53.041826 19302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:41:53.041939 19302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:41:53.047837 19302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:41:53.047916 19302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:41:53.048000 19302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:41:53.055663 19302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.222.212:35973/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:35703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:41:53.057050 19302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:41:53.058423 19302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:41:53.073449 19307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:53.073449 19308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:41:53.074096 19310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:41:53.075433 19302 server_base.cc:1034] running on GCE node
I20260809 04:41:53.075942 19302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:41:53.076761 19302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:41:53.092286 19302 hybrid_clock.cc:648] HybridClock initialized: now 1786250513092235 us; error 70 us; skew 500 ppm
Aug 09 04:41:53 dist-test-slave-vxj2 krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250513, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:41:53.098671 19302 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:41:53.099941 19302 webserver.cc:460] Webserver started at http://127.5.222.195:46211/ using document root <none> and password file <none>
I20260809 04:41:53.100723 19302 fs_manager.cc:359] Metadata directory not provided
I20260809 04:41:53.100862 19302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:41:53.101141 19302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:41:53 dist-test-slave-vxj2 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250513, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:41:53.131603 19302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "624c0b392ff64ad789273c0dbd9e7d0d"
format_stamp: "Formatted at 2026-08-09 04:41:53 on dist-test-slave-vxj2"
server_key: "7697591df9099314157c3b020e9c0b8f"
server_key_iv: "1b696bf928233ce51f85aa81bbcedf0a"
server_key_version: "kuduclusterkey@0"
I20260809 04:41:53.132567 19302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "624c0b392ff64ad789273c0dbd9e7d0d"
format_stamp: "Formatted at 2026-08-09 04:41:53 on dist-test-slave-vxj2"
server_key: "7697591df9099314157c3b020e9c0b8f"
server_key_iv: "1b696bf928233ce51f85aa81bbcedf0a"
server_key_version: "kuduclusterkey@0"
I20260809 04:41:53.138329 19302 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:41:53.159548 19318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:53.161387 19302 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:41:53.161513 19302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "624c0b392ff64ad789273c0dbd9e7d0d"
format_stamp: "Formatted at 2026-08-09 04:41:53 on dist-test-slave-vxj2"
server_key: "7697591df9099314157c3b020e9c0b8f"
server_key_iv: "1b696bf928233ce51f85aa81bbcedf0a"
server_key_version: "kuduclusterkey@0"
I20260809 04:41:53.161657 19302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:41:53.182772 19302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:41:53.186758 19302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:41:53.187011 19302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:41:53.187783 19302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:41:53.188908 19302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:41:53.189031 19302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:53.189126 19302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:41:53.189215 19302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:41:53.212669 19302 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:34125
I20260809 04:41:53.214092 19302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:41:53.216355 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 19302
I20260809 04:41:53.216478 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:41:53.216809 6011 external_mini_cluster.cc:1428] Setting key 5cbd7337d323b93e3f56112824b621a5
I20260809 04:41:53.219879 19445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:34125 every 8 connection(s)
Aug 09 04:41:53 dist-test-slave-vxj2 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250513, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:41:53.237843 19143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.221141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:55583 (local address 127.5.222.254:35703)
0809 04:41:53.221279 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:53.221284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:53.222269 (+ 985us) server_negotiation.cc:407] Connection header received
0809 04:41:53.223833 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:53.223837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:53.223889 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:53.223938 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:53.225739 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.226570 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.228382 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.228733 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.228810 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.233474 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:53.233478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:53.233481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:41:53.233510 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:53.235378 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:53.236016 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:53.236020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:53.236021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:53.236106 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:53.236510 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:53.236513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:53.236515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:53.236753 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:53.236832 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:53.237539 (+ 707us) server_negotiation.cc:299] Negotiation successful
0809 04:41:53.237669 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:41:53.239991 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.219998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:35703 (local address 127.5.222.195:55583)
0809 04:41:53.222114 (+ 2116us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:53.222172 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:53.223580 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:53.224340 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:53.224349 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:53.224813 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:41:53.225561 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:53.225580 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.226821 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.226825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:53.228241 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:53.228252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.229017 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.229021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:53.229315 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.230166 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:41:53.230187 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:41:53.233290 (+ 3103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:41:53.235528 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:41:53.235534 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:41:53.235551 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:41:53.235901 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:41:53.236203 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:41:53.236205 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:41:53.236207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:41:53.236394 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:41:53.236899 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:41:53.236905 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:41:53.237256 (+ 351us) client_negotiation.cc:769] Sending connection context
0809 04:41:53.238388 (+ 1132us) client_negotiation.cc:240] Negotiation successful
0809 04:41:53.238789 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1620,"thread_start_us":226,"threads_started":1}
I20260809 04:41:53.241541 19446 heartbeater.cc:347] Connected to a master server at 127.5.222.254:35703
I20260809 04:41:53.241792 19446 heartbeater.cc:464] Registering TS with master...
I20260809 04:41:53.242342 19446 heartbeater.cc:511] Master 127.5.222.254:35703 requested a full tablet report, sending...
I20260809 04:41:53.243484 18903 ts_manager.cc:194] Registered new tserver with Master: 624c0b392ff64ad789273c0dbd9e7d0d (127.5.222.195:34125)
I20260809 04:41:53.244222 18903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:55583
Aug 09 04:41:53 dist-test-slave-vxj2 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250512, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:41:53.247094 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:41:53.260493 19143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.249449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56350 (local address 127.5.222.254:35703)
0809 04:41:53.249563 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:53.249568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:53.250405 (+ 837us) server_negotiation.cc:407] Connection header received
0809 04:41:53.250661 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:53.250664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:53.250716 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:53.250770 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:41:53.251912 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.252715 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.254088 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.254481 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.254597 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.256568 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:41:53.256572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:41:53.256575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:41:53.256600 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:41:53.258496 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:53.259013 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:53.259016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:53.259018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:53.259105 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:41:53.259450 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:41:53.259454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:41:53.259455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:41:53.259663 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:41:53.259767 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:53.260147 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 04:41:53.260305 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:41:53.264209 18903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:41:53.270220 18903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:41:53.328397 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.308636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:34125 (local address 127.0.0.1:33852)
0809 04:41:53.313493 (+ 4857us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:53.313552 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:53.313703 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:53.318257 (+ 4554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:53.318265 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:53.318307 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:41:53.318477 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:53.318485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.324953 (+ 6468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.324957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:53.326197 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:53.326209 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.327807 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.327810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:53.327917 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.327955 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 04:41:53.328086 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 04:41:53.328232 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4655,"thread_start_us":178,"threads_started":1}
I20260809 04:41:53.333765 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.310505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:40147 (local address 127.0.0.1:60184)
0809 04:41:53.318726 (+ 8221us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:53.318742 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:53.318835 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:53.320896 (+ 2061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:53.320899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:53.320918 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:41:53.321041 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:53.321057 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.322610 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.322614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:53.324197 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:53.324208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.333367 (+ 9159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.333370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:53.333450 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.333471 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:41:53.333538 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 04:41:53.333602 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8141,"thread_start_us":159,"threads_started":1}
I20260809 04:41:53.339439 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.311366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:39015 (local address 127.0.0.1:39800)
0809 04:41:53.328851 (+ 17485us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:53.328865 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:53.328944 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:53.329740 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:53.329745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:53.329766 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:41:53.329851 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:53.329857 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.331230 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.331234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:53.337480 (+ 6246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:53.337491 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.339005 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.339009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:53.339101 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.339127 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:41:53.339206 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 04:41:53.339289 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17424,"thread_start_us":230,"threads_started":1}
I20260809 04:41:53.340061 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.309599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33852 (local address 127.5.222.195:34125)
0809 04:41:53.317404 (+ 7805us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:53.317417 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:53.317445 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:41:53.317534 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:53.317540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:53.317827 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:53.318018 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:41:53.319163 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.324803 (+ 5640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.326387 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.327423 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.327537 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.327708 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:53.339751 (+ 12043us) server_negotiation.cc:299] Negotiation successful
0809 04:41:53.339907 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7680,"thread_start_us":196,"threads_started":1}
I20260809 04:41:53.343830 19374 tablet_service.cc:1467] Processing CreateTablet for tablet 621b0dbb5cec4a6cbbff59f820b797a1 (DEFAULT_TABLE table=test-table [id=e9a859f513824032b0bd26c149f6ab0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:41:53.345170 19374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 621b0dbb5cec4a6cbbff59f820b797a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:41:53.353696 19467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.310689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60184 (local address 127.5.222.194:40147)
0809 04:41:53.320231 (+ 9542us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:53.320238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:53.320264 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:41:53.320340 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:53.320345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:53.320535 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:53.320691 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:41:53.321349 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.322450 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.332262 (+ 9812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.333025 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.333134 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.333304 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:53.353284 (+ 19980us) server_negotiation.cc:299] Negotiation successful
0809 04:41:53.353454 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9414,"thread_start_us":102,"threads_started":1}
I20260809 04:41:53.357275 19201 tablet_service.cc:1467] Processing CreateTablet for tablet 621b0dbb5cec4a6cbbff59f820b797a1 (DEFAULT_TABLE table=test-table [id=e9a859f513824032b0bd26c149f6ab0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:41:53.358884 19201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 621b0dbb5cec4a6cbbff59f820b797a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:41:53.365799 19468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.311512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39800 (local address 127.5.222.193:39015)
0809 04:41:53.319091 (+ 7579us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:53.319098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:53.329085 (+ 9987us) server_negotiation.cc:407] Connection header received
0809 04:41:53.329222 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:53.329228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:53.329444 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:53.329590 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:41:53.329991 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.331082 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.337641 (+ 6559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.338556 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.338664 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.338856 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:53.365420 (+ 26564us) server_negotiation.cc:299] Negotiation successful
0809 04:41:53.365587 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7466,"thread_start_us":73,"threads_started":1}
I20260809 04:41:53.377171 19064 tablet_service.cc:1467] Processing CreateTablet for tablet 621b0dbb5cec4a6cbbff59f820b797a1 (DEFAULT_TABLE table=test-table [id=e9a859f513824032b0bd26c149f6ab0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:41:53.378438 19064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 621b0dbb5cec4a6cbbff59f820b797a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:41:53.389496 19471 tablet_bootstrap.cc:492] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca: Bootstrap starting.
I20260809 04:41:53.392058 19471 tablet_bootstrap.cc:654] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca: Neither blocks nor log segments found. Creating new log.
I20260809 04:41:53.392961 19471 log.cc:824] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca: Log is configured to *not* fsync() on all Append() calls
I20260809 04:41:53.395162 19472 tablet_bootstrap.cc:492] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38: Bootstrap starting.
I20260809 04:41:53.396816 19471 tablet_bootstrap.cc:492] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca: No bootstrap required, opened a new log
I20260809 04:41:53.397043 19471 ts_tablet_manager.cc:1397] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260809 04:41:53.398087 19472 tablet_bootstrap.cc:654] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38: Neither blocks nor log segments found. Creating new log.
I20260809 04:41:53.399432 19139 heartbeater.cc:503] Master 127.5.222.254:35703 was elected leader, sending a full tablet report...
I20260809 04:41:53.399878 19472 log.cc:824] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38: Log is configured to *not* fsync() on all Append() calls
I20260809 04:41:53.400774 19474 tablet_bootstrap.cc:492] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d: Bootstrap starting.
I20260809 04:41:53.404526 19474 tablet_bootstrap.cc:654] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d: Neither blocks nor log segments found. Creating new log.
I20260809 04:41:53.405597 19471 raft_consensus.cc:348] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624c0b392ff64ad789273c0dbd9e7d0d" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34125 } } peers { permanent_uuid: "c97d086ca3034755a36ea3feec06edca" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 40147 } } peers { permanent_uuid: "6bc356e377a94e67839632bd52f1ca38" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 39015 } }
I20260809 04:41:53.405746 19474 log.cc:824] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:41:53.405855 19471 raft_consensus.cc:374] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:41:53.405917 19471 raft_consensus.cc:729] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97d086ca3034755a36ea3feec06edca, State: Initialized, Role: FOLLOWER
I20260809 04:41:53.406375 19471 consensus_queue.cc:260] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624c0b392ff64ad789273c0dbd9e7d0d" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34125 } } peers { permanent_uuid: "c97d086ca3034755a36ea3feec06edca" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 40147 } } peers { permanent_uuid: "6bc356e377a94e67839632bd52f1ca38" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 39015 } }
I20260809 04:41:53.407616 19471 ts_tablet_manager.cc:1428] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260809 04:41:53.407971 19297 heartbeater.cc:503] Master 127.5.222.254:35703 was elected leader, sending a full tablet report...
I20260809 04:41:53.408228 19472 tablet_bootstrap.cc:492] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38: No bootstrap required, opened a new log
I20260809 04:41:53.408398 19474 tablet_bootstrap.cc:492] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d: No bootstrap required, opened a new log
I20260809 04:41:53.408416 19472 ts_tablet_manager.cc:1397] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260809 04:41:53.408560 19474 ts_tablet_manager.cc:1397] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 04:41:53.414872 19472 raft_consensus.cc:348] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624c0b392ff64ad789273c0dbd9e7d0d" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34125 } } peers { permanent_uuid: "c97d086ca3034755a36ea3feec06edca" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 40147 } } peers { permanent_uuid: "6bc356e377a94e67839632bd52f1ca38" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 39015 } }
I20260809 04:41:53.415190 19472 raft_consensus.cc:374] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:41:53.415256 19472 raft_consensus.cc:729] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bc356e377a94e67839632bd52f1ca38, State: Initialized, Role: FOLLOWER
I20260809 04:41:53.415169 19474 raft_consensus.cc:348] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624c0b392ff64ad789273c0dbd9e7d0d" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34125 } } peers { permanent_uuid: "c97d086ca3034755a36ea3feec06edca" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 40147 } } peers { permanent_uuid: "6bc356e377a94e67839632bd52f1ca38" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 39015 } }
I20260809 04:41:53.415526 19474 raft_consensus.cc:374] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:41:53.415584 19474 raft_consensus.cc:729] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 624c0b392ff64ad789273c0dbd9e7d0d, State: Initialized, Role: FOLLOWER
I20260809 04:41:53.415789 19472 consensus_queue.cc:260] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624c0b392ff64ad789273c0dbd9e7d0d" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34125 } } peers { permanent_uuid: "c97d086ca3034755a36ea3feec06edca" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 40147 } } peers { permanent_uuid: "6bc356e377a94e67839632bd52f1ca38" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 39015 } }
I20260809 04:41:53.415982 19474 consensus_queue.cc:260] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624c0b392ff64ad789273c0dbd9e7d0d" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34125 } } peers { permanent_uuid: "c97d086ca3034755a36ea3feec06edca" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 40147 } } peers { permanent_uuid: "6bc356e377a94e67839632bd52f1ca38" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 39015 } }
I20260809 04:41:53.417066 19474 ts_tablet_manager.cc:1428] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d: Time spent starting tablet: real 0.008s user 0.004s sys 0.005s
I20260809 04:41:53.417104 19472 ts_tablet_manager.cc:1428] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260809 04:41:53.417497 19446 heartbeater.cc:503] Master 127.5.222.254:35703 was elected leader, sending a full tablet report...
W20260809 04:41:53.468072 19448 tablet.cc:2369] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:41:53.512399 19479 raft_consensus.cc:482] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:41:53.512701 19479 raft_consensus.cc:504] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624c0b392ff64ad789273c0dbd9e7d0d" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34125 } } peers { permanent_uuid: "c97d086ca3034755a36ea3feec06edca" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 40147 } } peers { permanent_uuid: "6bc356e377a94e67839632bd52f1ca38" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 39015 } }
I20260809 04:41:53.514061 19479 leader_election.cc:288] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 624c0b392ff64ad789273c0dbd9e7d0d (127.5.222.195:34125), c97d086ca3034755a36ea3feec06edca (127.5.222.194:40147)
I20260809 04:41:53.520718 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.514609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:56669 (local address 127.5.222.195:34125)
0809 04:41:53.514749 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:53.514754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:53.515700 (+ 946us) server_negotiation.cc:407] Connection header received
0809 04:41:53.515977 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:53.515980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:53.516035 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:53.516093 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:41:53.516631 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.517426 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.519189 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.519774 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.519852 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.519927 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:53.520504 (+ 577us) server_negotiation.cc:299] Negotiation successful
0809 04:41:53.520584 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:41:53.520718 19467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.514953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:39145 (local address 127.5.222.194:40147)
0809 04:41:53.515099 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:53.515103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:53.515120 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:41:53.515344 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:53.515348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:53.515404 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:53.515488 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:41:53.516132 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.516982 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.519025 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.519697 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.519797 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.519855 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:53.520504 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 04:41:53.520584 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:41:53.521130 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.514382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:34125 (local address 127.5.222.193:56669)
0809 04:41:53.515591 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:53.515607 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:53.515784 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:53.516227 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:53.516229 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:53.516245 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:41:53.516491 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:53.516498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.517655 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.517658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:53.519065 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:53.519074 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.520002 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.520006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:53.520310 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.520369 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 04:41:53.520861 (+ 492us) client_negotiation.cc:240] Negotiation successful
0809 04:41:53.520972 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1143,"mutex_wait_us":801,"thread_start_us":143,"threads_started":1}
I20260809 04:41:53.521657 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.514382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:40147 (local address 127.5.222.193:39145)
0809 04:41:53.515017 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:53.515035 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:41:53.515156 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:41:53.515680 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:41:53.515686 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:41:53.515707 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:41:53.515991 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:53.515999 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.517120 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.517123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:53.518858 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:41:53.518868 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.519958 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.519964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:41:53.520342 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.520380 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 04:41:53.521377 (+ 997us) client_negotiation.cc:240] Negotiation successful
0809 04:41:53.521445 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"mutex_wait_us":156,"thread_start_us":170,"threads_started":1}
I20260809 04:41:53.522004 19395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621b0dbb5cec4a6cbbff59f820b797a1" candidate_uuid: "6bc356e377a94e67839632bd52f1ca38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "624c0b392ff64ad789273c0dbd9e7d0d" is_pre_election: true
I20260809 04:41:53.522346 19395 raft_consensus.cc:2454] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bc356e377a94e67839632bd52f1ca38 in term 0.
I20260809 04:41:53.522475 19242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621b0dbb5cec4a6cbbff59f820b797a1" candidate_uuid: "6bc356e377a94e67839632bd52f1ca38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97d086ca3034755a36ea3feec06edca" is_pre_election: true
I20260809 04:41:53.522779 19242 raft_consensus.cc:2454] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bc356e377a94e67839632bd52f1ca38 in term 0.
W20260809 04:41:53.522961 19298 tablet.cc:2369] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:41:53.523022 19014 leader_election.cc:302] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 624c0b392ff64ad789273c0dbd9e7d0d, 6bc356e377a94e67839632bd52f1ca38; no voters:
I20260809 04:41:53.523348 19479 raft_consensus.cc:2781] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:41:53.523458 19479 raft_consensus.cc:482] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:41:53.523516 19479 raft_consensus.cc:3037] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:41:53.525525 19479 raft_consensus.cc:504] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624c0b392ff64ad789273c0dbd9e7d0d" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34125 } } peers { permanent_uuid: "c97d086ca3034755a36ea3feec06edca" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 40147 } } peers { permanent_uuid: "6bc356e377a94e67839632bd52f1ca38" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 39015 } }
I20260809 04:41:53.525964 19479 leader_election.cc:288] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [CANDIDATE]: Term 1 election: Requested vote from peers 624c0b392ff64ad789273c0dbd9e7d0d (127.5.222.195:34125), c97d086ca3034755a36ea3feec06edca (127.5.222.194:40147)
I20260809 04:41:53.526536 19395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621b0dbb5cec4a6cbbff59f820b797a1" candidate_uuid: "6bc356e377a94e67839632bd52f1ca38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "624c0b392ff64ad789273c0dbd9e7d0d"
I20260809 04:41:53.526667 19395 raft_consensus.cc:3037] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:41:53.528417 19395 raft_consensus.cc:2454] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bc356e377a94e67839632bd52f1ca38 in term 1.
I20260809 04:41:53.528756 19014 leader_election.cc:302] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 624c0b392ff64ad789273c0dbd9e7d0d, 6bc356e377a94e67839632bd52f1ca38; no voters:
I20260809 04:41:53.528801 19242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621b0dbb5cec4a6cbbff59f820b797a1" candidate_uuid: "6bc356e377a94e67839632bd52f1ca38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97d086ca3034755a36ea3feec06edca"
I20260809 04:41:53.528920 19242 raft_consensus.cc:3037] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:41:53.529012 19479 raft_consensus.cc:2781] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:41:53.529311 19479 raft_consensus.cc:686] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [term 1 LEADER]: Becoming Leader. State: Replica: 6bc356e377a94e67839632bd52f1ca38, State: Running, Role: LEADER
I20260809 04:41:53.529697 19479 consensus_queue.cc:237] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624c0b392ff64ad789273c0dbd9e7d0d" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34125 } } peers { permanent_uuid: "c97d086ca3034755a36ea3feec06edca" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 40147 } } peers { permanent_uuid: "6bc356e377a94e67839632bd52f1ca38" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 39015 } }
I20260809 04:41:53.530421 19242 raft_consensus.cc:2454] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bc356e377a94e67839632bd52f1ca38 in term 1.
I20260809 04:41:53.534490 18901 catalog_manager.cc:5495] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bc356e377a94e67839632bd52f1ca38 (127.5.222.193). New cstate: current_term: 1 leader_uuid: "6bc356e377a94e67839632bd52f1ca38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624c0b392ff64ad789273c0dbd9e7d0d" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97d086ca3034755a36ea3feec06edca" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 40147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bc356e377a94e67839632bd52f1ca38" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 39015 } health_report { overall_health: HEALTHY } } }
I20260809 04:41:53.547967 19468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.543866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39804 (local address 127.5.222.193:39015)
0809 04:41:53.543986 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 04:41:53.543991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:41:53.544051 (+ 60us) server_negotiation.cc:407] Connection header received
0809 04:41:53.544248 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:41:53.544254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:41:53.544333 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:41:53.544458 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:41:53.544707 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.545405 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.546352 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:41:53.546597 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:41:53.546710 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:41:53.547014 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:41:53.547489 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:41:53.547617 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 04:41:53.547657 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 04:41:53.547718 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 04:41:53.554260 19242 raft_consensus.cc:1261] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca [term 1 FOLLOWER]: Refusing update from remote peer 6bc356e377a94e67839632bd52f1ca38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:41:53.554260 19395 raft_consensus.cc:1261] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d [term 1 FOLLOWER]: Refusing update from remote peer 6bc356e377a94e67839632bd52f1ca38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:41:53.555168 19479 consensus_queue.cc:1035] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "624c0b392ff64ad789273c0dbd9e7d0d" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:41:53.557459 19479 consensus_queue.cc:1035] T 621b0dbb5cec4a6cbbff59f820b797a1 P 6bc356e377a94e67839632bd52f1ca38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97d086ca3034755a36ea3feec06edca" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 40147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:41:53.568532 19489 mvcc.cc:205] Tried to move back new op lower bound from 7316482103510802432 to 7316482103424741376. Current Snapshot: MvccSnapshot[applied={T|T < 7316482103510802432}]
I20260809 04:41:53.571717 19487 mvcc.cc:205] Tried to move back new op lower bound from 7316482103510802432 to 7316482103424741376. Current Snapshot: MvccSnapshot[applied={T|T < 7316482103510802432}]
I20260809 04:41:53.583633 18901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:41:53.583856 18901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:41:53.586920 18901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3722aaa469944960882b12e51afbd074: Sending DeleteTablet for 3 replicas of tablet 621b0dbb5cec4a6cbbff59f820b797a1
I20260809 04:41:53.588310 19374 tablet_service.cc:1514] Processing DeleteTablet for tablet 621b0dbb5cec4a6cbbff59f820b797a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:41:53 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:33852
I20260809 04:41:53.588888 19201 tablet_service.cc:1514] Processing DeleteTablet for tablet 621b0dbb5cec4a6cbbff59f820b797a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:41:53 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:60184
I20260809 04:41:53.589948 19064 tablet_service.cc:1514] Processing DeleteTablet for tablet 621b0dbb5cec4a6cbbff59f820b797a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:41:53 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:39800
I20260809 04:41:53.591113 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 18991
I20260809 04:41:53.594763 19496 tablet_replica.cc:330] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d: stopping tablet replica
I20260809 04:41:53.595033 19497 tablet_replica.cc:330] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca: stopping tablet replica
I20260809 04:41:53.595497 19497 raft_consensus.cc:2229] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:41:53.595865 19496 raft_consensus.cc:2229] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:41:53.596282 19496 raft_consensus.cc:2258] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:41:53.596601 19497 raft_consensus.cc:2258] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:41:53.598491 19497 ts_tablet_manager.cc:1905] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:41:53.601891 19497 ts_tablet_manager.cc:1918] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:41:53.602054 19497 log.cc:1196] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/621b0dbb5cec4a6cbbff59f820b797a1
I20260809 04:41:53.602450 19497 ts_tablet_manager.cc:1939] T 621b0dbb5cec4a6cbbff59f820b797a1 P c97d086ca3034755a36ea3feec06edca: Deleting consensus metadata
I20260809 04:41:53.604043 18885 catalog_manager.cc:4841] TS c97d086ca3034755a36ea3feec06edca (127.5.222.194:40147): tablet 621b0dbb5cec4a6cbbff59f820b797a1 (table test-table [id=e9a859f513824032b0bd26c149f6ab0b]) successfully deleted
W20260809 04:41:53.604535 18886 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:39015 (error 108)
I20260809 04:41:53.605665 19496 ts_tablet_manager.cc:1905] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:41:53.605710 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 19149
I20260809 04:41:53.610069 19496 ts_tablet_manager.cc:1918] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:41:53.610211 19496 log.cc:1196] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/621b0dbb5cec4a6cbbff59f820b797a1
I20260809 04:41:53.610539 19496 ts_tablet_manager.cc:1939] T 621b0dbb5cec4a6cbbff59f820b797a1 P 624c0b392ff64ad789273c0dbd9e7d0d: Deleting consensus metadata
I20260809 04:41:53.611809 18888 catalog_manager.cc:4841] TS 624c0b392ff64ad789273c0dbd9e7d0d (127.5.222.195:34125): tablet 621b0dbb5cec4a6cbbff59f820b797a1 (table test-table [id=e9a859f513824032b0bd26c149f6ab0b]) successfully deleted
I20260809 04:41:53.612993 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:41:53.612620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:39015 (local address 127.0.0.1:39814)
0809 04:41:53.612809 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 04:41:53.612895 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:39015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260809 04:41:53.613303 18886 catalog_manager.cc:4584] TS 6bc356e377a94e67839632bd52f1ca38 (127.5.222.193:39015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 621b0dbb5cec4a6cbbff59f820b797a1: Network error: Client connection negotiation failed: client connection to 127.5.222.193:39015: connect: Connection refused (error 111)
I20260809 04:41:53.616631 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 19302
I20260809 04:41:53.626279 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 18869
I20260809 04:41:53.635087 6011 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 04:41:53.979817 6011 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 04:41:53.979984 6011 mini_ranger.cc:66] Stopping Ranger...
I20260809 04:41:54.324810 6011 mini_ranger.cc:68] Stopped Ranger
2026-08-09 04:41:54.325 UTC [18409] LOG: received smart shutdown request
2026-08-09 04:41:54.328 UTC [18409] LOG: background worker "logical replication launcher" (PID 18419) exited with exit code 1
2026-08-09 04:41:54.329 UTC [18414] LOG: shutting down
2026-08-09 04:41:54.512 UTC [18409] LOG: database system is shut down
2026-08-09T04:41:54Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47611 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:41:54 dist-test-slave-vxj2 krb5kdc[19511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:41:54 dist-test-slave-vxj2 krb5kdc[19511](info): set up 2 sockets
Aug 09 04:41:54 dist-test-slave-vxj2 krb5kdc[19511](info): commencing operation
krb5kdc: starting...
W20260809 04:41:55.748200 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:41:55 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:41:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:41:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.222.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.222.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/rangeradmin_127.5.222.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/rangeradmin_127.5.222.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.222.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.222.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/rangerlookup_127.5.222.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/rangerlookup_127.5.222.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.222.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.222.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/HTTP_127.5.222.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/HTTP_127.5.222.212@KRBTEST.COM.keytab.
I20260809 04:41:55.917488 6011 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskxJEoz3/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/postgres -l logfile start
2026-08-09 04:41:58.455 UTC [19546] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 04:41:58.455 UTC [19546] LOG: listening on IPv4 address "127.5.222.212", port 34267
2026-08-09 04:41:58.459 UTC [19546] LOG: listening on Unix socket "/tmp/.s.PGSQL.34267"
2026-08-09 04:41:58.482 UTC [19549] LOG: database system was shut down at 2026-08-09 04:41:57 UTC
2026-08-09 04:41:58.486 UTC [19546] LOG: database system is ready to accept connections
I20260809 04:41:59.492484 6011 mini_postgres.cc:96] Postgres bound to 34267
2026-08-09 04:41:59.499 UTC [19559] FATAL: database "slave" does not exist
127.5.222.212:34267 - accepting connections
I20260809 04:41:59.499843 6011 mini_ranger.cc:161] Starting Ranger...
I20260809 04:41:59.515019 6011 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 04:41:59.994655 6011 mini_ranger.cc:87] Created ranger Postgres database
I20260809 04:41:59.994752 6011 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-admin
2026-08-09 04:42:00,285 [I] DB FLAVOR :POSTGRES
2026-08-09 04:42:00,286 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:42:00,286 [I] Checking connection..
2026-08-09 04:42:00,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 04:42:00,553 [I] Checking connection passed.
2026-08-09 04:42:00,553 [I] --------- Verifying version history table ---------
2026-08-09 04:42:00,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:42:00,788 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:42:00,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:42:01,054 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:42:01,055 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 04:42:01,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 04:42:01.319 UTC [19632] WARNING: there is no transaction in progress
2026-08-09 04:42:01,326 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 04:42:01,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:42:01,587 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 04:42:01,588 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:42:01,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 04:42:01,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 04:42:02,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vxj2.c.gcp-upstream.internal','N') ;"
2026-08-09 04:42:02,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:42:02,584 [I] Table x_portal_user does not exist in database ranger
2026-08-09 04:42:02,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:42:02,841 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 04:42:02,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:42:03,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:42:03,348 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 04:42:03,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 04:42:04.322 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.338 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.355 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.371 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.390 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.407 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.461 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.467 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.482 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.492 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.502 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.517 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.526 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.533 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.540 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:04.546 UTC [19752] WARNING: there is no transaction in progress
2026-08-09 04:42:05,198 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 04:42:05,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 04:42:05,433 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 04:42:05,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:42:05,700 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 04:42:05,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:42:05,988 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 04:42:05,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:42:06,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:42:06,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vxj2.c.gcp-upstream.internal';"
2026-08-09 04:42:06,725 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 04:42:06,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 04:42:06,986 [I] DB_PATCHES have already been applied
I20260809 04:42:06.992481 6011 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-admin:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskxJEoz3/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 04:42:06.992559 6011 mini_ranger.cc:193] Using host: 127.5.222.212
I20260809 04:42:06.996464 6011 mini_ranger.cc:227] Ranger admin URL: http://127.5.222.212:36713
Aug 09, 2026 4:42:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:42:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:42:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:42:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:42:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:42:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:42:07 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 04:42:08.191444 6011 mini_ranger.cc:160] Time spent starting Ranger: real 8.692s user 0.000s sys 0.011s
Aug 09, 2026 4:42:28 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:42:28 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 4:42:28 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:42:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:42:28 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:42:29 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 04:42:30.414567 6011 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.222.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.222.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/rangerkms_127.5.222.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.222.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/rangerkms_127.5.222.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.222.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.222.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.222.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/HTTP_127.5.222.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.222.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/HTTP_127.5.222.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 04:42:30 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250550, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 04:42:30.554744 6011 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 04:42:30.575371 6011 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 04:42:31.433602 6011 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 04:42:31.433678 6011 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-kms
I20260809 04:42:31.433691 6011 mini_ranger_kms.cc:227] Using postgres at 127.5.222.212:34267
2026-08-09 04:42:31,616 [I] DB FLAVOR :POSTGRES
2026-08-09 04:42:31,616 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:42:31,616 [I] Checking connection
2026-08-09 04:42:31,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 04:42:31,875 [I] connection success
2026-08-09 04:42:31,875 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 04:42:31,875 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 04:42:31,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 04:42:32,113 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 04:42:32,113 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:42:32,113 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 04:42:32,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.222.212:34267/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 04:42:32,385 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 04:42:32.887585 6011 mini_ranger_kms.cc:310] Created kms service
I20260809 04:42:33.099457 6011 mini_ranger_kms.cc:326] Created kudu user
I20260809 04:42:33.143122 6011 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 04:42:33 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250550, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:42:33.541270 6011 mini_ranger_kms.cc:384] Added ranger policy
I20260809 04:42:33.541437 6011 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-kms:/tmp/dist-test-taskxJEoz3/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskxJEoz3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 04:42:33.541460 6011 mini_ranger_kms.cc:240] Using host: 127.5.222.212
I20260809 04:42:33.546236 6011 mini_ranger_kms.cc:276] Ranger KMS PID: 19947
I20260809 04:42:33.546334 6011 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.222.212:59427
Aug 09, 2026 4:42:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:42:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:42:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:42:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:42:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:42:35 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 04:42:36.012447 6011 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.458s user 0.013s sys 0.157s
I20260809 04:42:36.012600 6011 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 04:42:36.012687 6011 mini_ranger_kms.cc:401] 127.5.222.212:59427/kms/v1/keys
Aug 09 04:42:36 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250556, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.222.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 04:42:36 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250556, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.222.212@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
Aug 09 04:42:36 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250556, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.222.212@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
Aug 09 04:42:39 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:42:39.300783 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:41287
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:41287
--ranger_config_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.222.212:59427/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:42:39.413096 20004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:39.413363 20004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:39.413436 20004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:39.417376 20004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:42:39.417430 20004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:39.417449 20004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:39.417507 20004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:42:39.417537 20004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:42:39.422827 20004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.222.212:59427/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:41287
--ranger_config_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:41287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:39.423838 20004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:39.424839 20004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:39.433565 20010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:39.433612 20011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:39.433660 20014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:39.433583 20004 server_base.cc:1034] running on GCE node
I20260809 04:42:39.434370 20004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:39.435514 20004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:39.436704 20004 hybrid_clock.cc:648] HybridClock initialized: now 1786250559436685 us; error 29 us; skew 500 ppm
Aug 09 04:42:39 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250559, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:39.443934 20004 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:42:39.444797 20004 webserver.cc:460] Webserver started at http://127.5.222.254:41805/ using document root <none> and password file <none>
I20260809 04:42:39.445387 20004 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:39.445446 20004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:39.445616 20004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:42:39 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250559, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:42:39.497267 20004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "8c25491653d94a82a3d77b155f24b847"
format_stamp: "Formatted at 2026-08-09 04:42:39 on dist-test-slave-vxj2"
server_key: "05b847f24e5ad7c15d12deda07609292"
server_key_iv: "5ad344c0ab768bd54a8d4367e0934104"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:39.497932 20004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "8c25491653d94a82a3d77b155f24b847"
format_stamp: "Formatted at 2026-08-09 04:42:39 on dist-test-slave-vxj2"
server_key: "05b847f24e5ad7c15d12deda07609292"
server_key_iv: "5ad344c0ab768bd54a8d4367e0934104"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:39.502372 20004 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:42:39.521014 20023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:39.522475 20004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:42:39.522593 20004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "8c25491653d94a82a3d77b155f24b847"
format_stamp: "Formatted at 2026-08-09 04:42:39 on dist-test-slave-vxj2"
server_key: "05b847f24e5ad7c15d12deda07609292"
server_key_iv: "5ad344c0ab768bd54a8d4367e0934104"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:39.522768 20004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:39.551326 20004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:39.554359 20004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:39.554589 20004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:39.562894 20004 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:41287
I20260809 04:42:39.562904 20078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:41287 every 8 connection(s)
I20260809 04:42:39.563849 20004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:42:39.566627 20080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:39.568336 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 20004
I20260809 04:42:39.568463 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:42:39.568768 6011 external_mini_cluster.cc:1428] Setting key 2f926dd86470fdeb7738f4f02d4ab8b8
I20260809 04:42:39.573657 20080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847: Bootstrap starting.
I20260809 04:42:39.576126 20080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:39.577020 20080 log.cc:824] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:39.579099 20080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847: No bootstrap required, opened a new log
Aug 09 04:42:39 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:39.583316 20080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c25491653d94a82a3d77b155f24b847" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41287 } }
I20260809 04:42:39.583573 20080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:39.583645 20080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c25491653d94a82a3d77b155f24b847, State: Initialized, Role: FOLLOWER
I20260809 04:42:39.584136 20080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c25491653d94a82a3d77b155f24b847" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41287 } }
I20260809 04:42:39.584340 20080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:42:39.584456 20080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:42:39.584764 20080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:39.585594 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:39.570214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46938 (local address 127.5.222.254:41287)
0809 04:42:39.570689 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:39.570708 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:39.570763 (+ 55us) server_negotiation.cc:407] Connection header received
0809 04:42:39.571652 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:39.571674 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:39.572079 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:39.572449 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:39.573358 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:39.574758 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:39.576002 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:39.576691 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:39.576864 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:39.580117 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:39.580124 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:39.580140 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:39.580171 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:39.582331 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:39.582858 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:39.582864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:39.582870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:39.582999 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:39.583338 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:39.583342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:39.583344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:39.583844 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:39.584083 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:39.584365 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 04:42:39.584632 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":130,"threads_started":1}
I20260809 04:42:39.585800 20080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c25491653d94a82a3d77b155f24b847" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41287 } }
I20260809 04:42:39.586148 20080 leader_election.cc:302] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c25491653d94a82a3d77b155f24b847; no voters:
I20260809 04:42:39.586526 20080 leader_election.cc:288] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:42:39.586622 20086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:42:39.586867 20086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 1 LEADER]: Becoming Leader. State: Replica: 8c25491653d94a82a3d77b155f24b847, State: Running, Role: LEADER
I20260809 04:42:39.587206 20086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c25491653d94a82a3d77b155f24b847" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41287 } }
I20260809 04:42:39.587872 20080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:42:39.588415 20086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c25491653d94a82a3d77b155f24b847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c25491653d94a82a3d77b155f24b847" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41287 } } }
I20260809 04:42:39.588526 20086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:39.588762 20086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c25491653d94a82a3d77b155f24b847. Latest consensus state: current_term: 1 leader_uuid: "8c25491653d94a82a3d77b155f24b847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c25491653d94a82a3d77b155f24b847" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41287 } } }
I20260809 04:42:39.588929 20086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:39.588935 20091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:42:39.592167 20091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:42:39.592779 20080 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 04:42:39.605110 20091 catalog_manager.cc:1347] Generated new cluster ID: 008f821a370747e9abcd7f22ef1b5e05
I20260809 04:42:39.605376 20091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:42:39.625254 20091 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:42:39.627149 20091 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:42:39.653028 20091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847: Generated new TSK 0
I20260809 04:42:39.654145 20091 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:42:40.321767 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.222.212:59427/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41287
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:42:40.481073 20128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:40.481547 20128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:40.481660 20128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:40.487051 20128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:40.487147 20128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:40.487280 20128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:42:40.493456 20128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.222.212:59427/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:40.494590 20128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:40.495687 20128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:40.505897 20136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:40.506026 20135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:40.505905 20138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:40.507261 20128 server_base.cc:1034] running on GCE node
I20260809 04:42:40.507702 20128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:40.508445 20128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:40.509639 20128 hybrid_clock.cc:648] HybridClock initialized: now 1786250560509601 us; error 50 us; skew 500 ppm
Aug 09 04:42:40 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250560, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:40.512445 20128 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:42:40.513463 20128 webserver.cc:460] Webserver started at http://127.5.222.193:38419/ using document root <none> and password file <none>
I20260809 04:42:40.514058 20128 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:40.514151 20128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:40.514370 20128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:42:40 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250560, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:42:40.529357 20128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "d46210f91c4c46a2b944e377fc024913"
format_stamp: "Formatted at 2026-08-09 04:42:40 on dist-test-slave-vxj2"
server_key: "fb84e7d4e1ede8f29b12092c5850b09f"
server_key_iv: "c2aa464f0ff3138f668802e21ef67502"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:40.530310 20128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "d46210f91c4c46a2b944e377fc024913"
format_stamp: "Formatted at 2026-08-09 04:42:40 on dist-test-slave-vxj2"
server_key: "fb84e7d4e1ede8f29b12092c5850b09f"
server_key_iv: "c2aa464f0ff3138f668802e21ef67502"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:40.534361 20128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:42:40.565517 20146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:40.567219 20128 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.001s sys 0.000s
I20260809 04:42:40.567359 20128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "d46210f91c4c46a2b944e377fc024913"
format_stamp: "Formatted at 2026-08-09 04:42:40 on dist-test-slave-vxj2"
server_key: "fb84e7d4e1ede8f29b12092c5850b09f"
server_key_iv: "c2aa464f0ff3138f668802e21ef67502"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:40.567489 20128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:40.598537 20128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:40.602243 20128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:40.602512 20128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:40.603302 20128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:40.604396 20128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:42:40.604507 20128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:40.604635 20128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:42:40.604720 20128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:40.616432 20128 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:32867
I20260809 04:42:40.617806 20128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:42:40.618243 20273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:32867 every 8 connection(s)
I20260809 04:42:40.621214 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 20128
I20260809 04:42:40.621344 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:42:40.621647 6011 external_mini_cluster.cc:1428] Setting key d1aecdfecbc7c2d8b1382306727a9ab5
Aug 09 04:42:40 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250560, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:42:40.639256 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:40.621112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41287 (local address 127.5.222.193:59767)
0809 04:42:40.622369 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:40.622425 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:40.623407 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:40.624282 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:40.624291 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:40.624651 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:40.625432 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:40.625455 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:40.626667 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:40.626671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:40.628145 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:40.628156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:40.628867 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:40.628871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:40.629060 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:40.629803 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:40.629825 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:40.631608 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:40.634002 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:40.634008 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:40.634024 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:40.634376 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:40.634740 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:40.634744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:40.634746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:40.634942 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:40.635438 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:40.635445 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:40.635768 (+ 323us) client_negotiation.cc:769] Sending connection context
0809 04:42:40.635992 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 04:42:40.636397 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"thread_start_us":145,"threads_started":1}
I20260809 04:42:40.640084 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:40.622261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:59767 (local address 127.5.222.254:41287)
0809 04:42:40.623825 (+ 1564us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:40.623836 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:40.623852 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:42:40.623894 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:40.623899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:40.623951 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:40.624048 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:40.625623 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:40.626524 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:40.628300 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:40.628657 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:40.629440 (+ 783us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:40.631823 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:40.631827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:40.631831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:40.631858 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:40.633861 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:40.634508 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:40.634512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:40.634514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:40.634611 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:40.635068 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:40.635072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:40.635074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:40.635305 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:40.639655 (+ 4350us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:40.639759 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 04:42:40.639907 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1480,"thread_start_us":113,"threads_started":1}
I20260809 04:42:40.641389 20274 heartbeater.cc:347] Connected to a master server at 127.5.222.254:41287
I20260809 04:42:40.641629 20274 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:40.642199 20274 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
I20260809 04:42:40.643976 20035 ts_manager.cc:194] Registered new tserver with Master: d46210f91c4c46a2b944e377fc024913 (127.5.222.193:32867)
I20260809 04:42:40.645563 20035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:59767
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:42:40.657613 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.222.212:59427/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41287
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:42:40 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250560, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 04:42:40.804579 20281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:40.805039 20281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:40.805184 20281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:40.811308 20281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:40.811456 20281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:40.811589 20281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:42:40.819516 20281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.222.212:59427/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:40.821130 20281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:40.822552 20281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:40.835136 20287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:40.835256 20289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:40.835549 20291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:40.835784 20281 server_base.cc:1034] running on GCE node
I20260809 04:42:40.836339 20281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:40.837090 20281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:40.838311 20281 hybrid_clock.cc:648] HybridClock initialized: now 1786250560838286 us; error 39 us; skew 500 ppm
Aug 09 04:42:40 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250560, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:40.841987 20281 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:42:40.843171 20281 webserver.cc:460] Webserver started at http://127.5.222.194:32873/ using document root <none> and password file <none>
I20260809 04:42:40.843896 20281 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:40.843992 20281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:40.844297 20281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:42:40 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250560, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:42:40.861079 20281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "5b6f3865a3c2400ab551c14af17bb59c"
format_stamp: "Formatted at 2026-08-09 04:42:40 on dist-test-slave-vxj2"
server_key: "22f24776dd98bae8623829365aea32fb"
server_key_iv: "6e853b728de217a0e739438724f985e4"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:40.861985 20281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "5b6f3865a3c2400ab551c14af17bb59c"
format_stamp: "Formatted at 2026-08-09 04:42:40 on dist-test-slave-vxj2"
server_key: "22f24776dd98bae8623829365aea32fb"
server_key_iv: "6e853b728de217a0e739438724f985e4"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:40.872025 20281 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260809 04:42:40.897895 20299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:40.902709 20281 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260809 04:42:40.902872 20281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "5b6f3865a3c2400ab551c14af17bb59c"
format_stamp: "Formatted at 2026-08-09 04:42:40 on dist-test-slave-vxj2"
server_key: "22f24776dd98bae8623829365aea32fb"
server_key_iv: "6e853b728de217a0e739438724f985e4"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:40.903043 20281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:40.931013 20281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:40.938974 20281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:40.939239 20281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:40.940130 20281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:40.941254 20281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:42:40.941339 20281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:40.941445 20281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:42:40.941520 20281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:40.955221 20281 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:46637
I20260809 04:42:40.956668 20281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:42:40.957469 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 20281
I20260809 04:42:40.957598 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:42:40.957944 6011 external_mini_cluster.cc:1428] Setting key 08d86d5cf7b290c24812031c70c018d1
I20260809 04:42:40.958388 20426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:46637 every 8 connection(s)
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
Aug 09 04:42:40 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250560, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:41.002332 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:40.964514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41287 (local address 127.5.222.194:48471)
0809 04:42:40.973319 (+ 8805us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:40.973395 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:40.974499 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:40.977477 (+ 2978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:40.977488 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:40.977869 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:40.978568 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:40.978589 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:40.984434 (+ 5845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:40.984438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:40.985972 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:40.985985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:40.986997 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:40.987001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:40.987338 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:40.994633 (+ 7295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:40.994658 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:40.996501 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:40.998739 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:40.998745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:40.998764 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:40.999121 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:40.999465 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:40.999469 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:40.999470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:40.999677 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:41.000191 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:41.000199 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:41.000561 (+ 362us) client_negotiation.cc:769] Sending connection context
0809 04:42:41.000780 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 04:42:41.001142 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8378,"thread_start_us":160,"threads_started":1}
I20260809 04:42:41.003077 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:40.964796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:48471 (local address 127.5.222.254:41287)
0809 04:42:40.964926 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:40.964931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:40.973509 (+ 8578us) server_negotiation.cc:407] Connection header received
0809 04:42:40.974746 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:40.974750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:40.974817 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:40.974912 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:40.978734 (+ 3822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:40.979612 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:40.986273 (+ 6661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:40.986677 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:40.986770 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:40.996756 (+ 9986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:40.996760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:40.996764 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:40.996797 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:40.998589 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:40.999246 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:40.999250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:40.999252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:40.999355 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:40.999797 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:40.999800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:40.999802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:41.000038 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:41.000135 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:41.002770 (+ 2635us) server_negotiation.cc:299] Negotiation successful
0809 04:42:41.002922 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 04:42:41.004912 20427 heartbeater.cc:347] Connected to a master server at 127.5.222.254:41287
I20260809 04:42:41.005182 20427 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:41.005788 20427 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
I20260809 04:42:41.007037 20035 ts_manager.cc:194] Registered new tserver with Master: 5b6f3865a3c2400ab551c14af17bb59c (127.5.222.194:46637)
I20260809 04:42:41.007876 20035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:48471
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:42:41.021230 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.222.212:59427/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41287
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:42:41 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250560, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
W20260809 04:42:41.287180 20432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:41.287719 20432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:41.287833 20432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:41.294502 20432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:41.294646 20432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:41.294785 20432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:42:41.302567 20432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.222.212:59427/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:41.304437 20432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:41.306177 20432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:41.319682 20437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:41.321627 20438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:41.323127 20440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:41.323235 20432 server_base.cc:1034] running on GCE node
I20260809 04:42:41.323879 20432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:41.324640 20432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:41.332118 20432 hybrid_clock.cc:648] HybridClock initialized: now 1786250561332039 us; error 91 us; skew 500 ppm
Aug 09 04:42:41 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250561, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:41.336146 20432 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:42:41.337419 20432 webserver.cc:460] Webserver started at http://127.5.222.195:37029/ using document root <none> and password file <none>
I20260809 04:42:41.338196 20432 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:41.338356 20432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:41.338650 20432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:42:41 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250561, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:42:41.367475 20432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "4985f51e10a0401a881ceb743244db01"
format_stamp: "Formatted at 2026-08-09 04:42:41 on dist-test-slave-vxj2"
server_key: "8e0299f1709e389b5822ac3cb5ebddf5"
server_key_iv: "06d1badac6a5d4f4e407d22d1aaa4530"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:41.371353 20432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "4985f51e10a0401a881ceb743244db01"
format_stamp: "Formatted at 2026-08-09 04:42:41 on dist-test-slave-vxj2"
server_key: "8e0299f1709e389b5822ac3cb5ebddf5"
server_key_iv: "06d1badac6a5d4f4e407d22d1aaa4530"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:41.376536 20432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:42:41.401542 20447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:41.403293 20432 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260809 04:42:41.403425 20432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "4985f51e10a0401a881ceb743244db01"
format_stamp: "Formatted at 2026-08-09 04:42:41 on dist-test-slave-vxj2"
server_key: "8e0299f1709e389b5822ac3cb5ebddf5"
server_key_iv: "06d1badac6a5d4f4e407d22d1aaa4530"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:41.403558 20432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:41.439626 20432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:41.442945 20432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:41.443164 20432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:41.443802 20432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:41.444784 20432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:42:41.444859 20432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:41.444948 20432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:42:41.444994 20432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:41.455997 20432 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:46239
I20260809 04:42:41.456938 20432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:42:41.457294 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 20432
I20260809 04:42:41.457419 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:42:41.457792 6011 external_mini_cluster.cc:1428] Setting key a428b3db5ab412b1720886169fc1f7df
I20260809 04:42:41.458616 20570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:46239 every 8 connection(s)
Aug 09 04:42:41 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250561, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:41.474020 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:41.461713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:49673 (local address 127.5.222.254:41287)
0809 04:42:41.461834 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:41.461838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:41.461852 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:42:41.462511 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:41.462514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:41.462568 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:41.462623 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:41.464078 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:41.464717 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:41.465904 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:41.466142 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:41.466256 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:41.469324 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:41.469328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:41.469331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:41.469358 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:41.471509 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:41.472140 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:41.472145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:41.472146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:41.472232 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:41.472664 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:41.472667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:41.472669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:41.472895 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:41.472982 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:41.473717 (+ 735us) server_negotiation.cc:299] Negotiation successful
0809 04:42:41.473854 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 04:42:41.475878 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:41.460678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41287 (local address 127.5.222.195:49673)
0809 04:42:41.461216 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:41.461280 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:41.462258 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:41.463029 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:41.463040 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:41.463334 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:41.463905 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:41.463927 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:41.464851 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:41.464854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:41.465782 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:41.465791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:41.466311 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:41.466316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:41.466492 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:41.467135 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:41.467157 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:41.469086 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:41.471636 (+ 2550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:41.471644 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:41.471661 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:41.472026 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:41.472363 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:41.472367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:41.472369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:41.472546 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:41.473053 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:41.473059 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:41.473419 (+ 360us) client_negotiation.cc:769] Sending connection context
0809 04:42:41.474450 (+ 1031us) client_negotiation.cc:240] Negotiation successful
0809 04:42:41.474803 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":160,"threads_started":1}
I20260809 04:42:41.477509 20572 heartbeater.cc:347] Connected to a master server at 127.5.222.254:41287
I20260809 04:42:41.477761 20572 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:41.478286 20572 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
I20260809 04:42:41.479348 20035 ts_manager.cc:194] Registered new tserver with Master: 4985f51e10a0401a881ceb743244db01 (127.5.222.195:46239)
I20260809 04:42:41.479967 20035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:49673
I20260809 04:42:41.484185 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:42:41.485010 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 20128
I20260809 04:42:41.500447 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 20281
I20260809 04:42:41.509529 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 20432
Aug 09 04:42:41 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250560, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:42:41.520432 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 20004
I20260809 04:42:41.533322 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:41287
--webserver_interface=127.5.222.254
--webserver_port=41805
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:41287
--ranger_config_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.222.212:59427/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.222.212:59427/kms with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:42:41.643571 20576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:41.643814 20576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:41.643857 20576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:41.647922 20576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:42:41.647975 20576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:41.647994 20576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:41.648010 20576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:42:41.648063 20576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:42:41.653447 20576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.222.212:59427/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:41287
--ranger_config_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:41287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=41805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:41.654403 20576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:41.655390 20576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:41.663731 20582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:41.663787 20581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:41.663741 20585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:41.664187 20576 server_base.cc:1034] running on GCE node
I20260809 04:42:41.664707 20576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:41.665840 20576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:41.667013 20576 hybrid_clock.cc:648] HybridClock initialized: now 1786250561666994 us; error 27 us; skew 500 ppm
Aug 09 04:42:41 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250561, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:41.669737 20576 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:42:41.670511 20576 webserver.cc:460] Webserver started at http://127.5.222.254:41805/ using document root <none> and password file <none>
I20260809 04:42:41.671008 20576 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:41.671072 20576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:42:41 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250561, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:42:41.750882 20576 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:42:41.753419 20595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:41.754873 20576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:42:41.755029 20576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "8c25491653d94a82a3d77b155f24b847"
format_stamp: "Formatted at 2026-08-09 04:42:39 on dist-test-slave-vxj2"
server_key: "05b847f24e5ad7c15d12deda07609292"
server_key_iv: "5ad344c0ab768bd54a8d4367e0934104"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:41.755635 20576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:41.788650 20576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:41.791522 20576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:41.791741 20576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:41.799467 20576 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:41287
I20260809 04:42:41.799471 20651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:41287 every 8 connection(s)
I20260809 04:42:41.800585 20576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:42:41.801702 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 20576
I20260809 04:42:41.802230 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.222.212:59427/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:32867
--local_ip_for_outbound_sockets=127.5.222.193
--tserver_master_addrs=127.5.222.254:41287
--webserver_port=38419
--webserver_interface=127.5.222.193
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.222.212:59427/kms with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:42:41.805811 20652 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:42:41.807524 20652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847: Bootstrap starting.
I20260809 04:42:41.823839 20652 log.cc:824] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:41.830498 20652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:42:41.830935 20652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847: Bootstrap complete.
I20260809 04:42:41.835331 20652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c25491653d94a82a3d77b155f24b847" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41287 } }
I20260809 04:42:41.835713 20652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c25491653d94a82a3d77b155f24b847, State: Initialized, Role: FOLLOWER
I20260809 04:42:41.836215 20652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c25491653d94a82a3d77b155f24b847" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41287 } }
I20260809 04:42:41.836331 20652 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:42:41.836434 20652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:42:41.836530 20652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:42:41.838470 20652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c25491653d94a82a3d77b155f24b847" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41287 } }
I20260809 04:42:41.838802 20652 leader_election.cc:302] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c25491653d94a82a3d77b155f24b847; no voters:
I20260809 04:42:41.839143 20652 leader_election.cc:288] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:42:41.839231 20656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:42:41.839457 20656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [term 2 LEADER]: Becoming Leader. State: Replica: 8c25491653d94a82a3d77b155f24b847, State: Running, Role: LEADER
I20260809 04:42:41.839808 20656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c25491653d94a82a3d77b155f24b847" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41287 } }
I20260809 04:42:41.840440 20652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:42:41.840765 20656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c25491653d94a82a3d77b155f24b847. Latest consensus state: current_term: 2 leader_uuid: "8c25491653d94a82a3d77b155f24b847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c25491653d94a82a3d77b155f24b847" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41287 } } }
I20260809 04:42:41.840881 20656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:41.841312 20657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8c25491653d94a82a3d77b155f24b847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c25491653d94a82a3d77b155f24b847" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41287 } } }
I20260809 04:42:41.842412 20657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847 [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:41.842811 20652 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 04:42:41.843403 20662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:42:41.849831 20662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:42:41.850687 20662 catalog_manager.cc:1259] Loaded cluster ID: 008f821a370747e9abcd7f22ef1b5e05
I20260809 04:42:41.850816 20662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:42:41.853669 20662 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:42:41.854557 20662 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847: Loaded TSK: 0
I20260809 04:42:41.856359 20662 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:42:41.951263 20654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:41.951697 20654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:41.951819 20654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:41.958005 20654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:41.958155 20654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:41.958304 20654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:42:41.966327 20654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.222.212:59427/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:32867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=38419
--tserver_master_addrs=127.5.222.254:41287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:41.967911 20654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:41.969230 20654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:41.981956 20693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:41.982481 20654 server_base.cc:1034] running on GCE node
W20260809 04:42:41.983033 20696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:41.983268 20694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:41.983798 20654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:41.984488 20654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:41.985679 20654 hybrid_clock.cc:648] HybridClock initialized: now 1786250561985648 us; error 45 us; skew 500 ppm
Aug 09 04:42:41 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250561, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:41.988601 20654 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:42:41.989709 20654 webserver.cc:460] Webserver started at http://127.5.222.193:38419/ using document root <none> and password file <none>
I20260809 04:42:41.990324 20654 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:41.990451 20654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:42:41 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250561, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:42:42.014137 20654 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.000s sys 0.003s
I20260809 04:42:42.017447 20706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:42.019101 20654 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 04:42:42.019228 20654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "d46210f91c4c46a2b944e377fc024913"
format_stamp: "Formatted at 2026-08-09 04:42:40 on dist-test-slave-vxj2"
server_key: "fb84e7d4e1ede8f29b12092c5850b09f"
server_key_iv: "c2aa464f0ff3138f668802e21ef67502"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:42.020016 20654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:42.032984 20654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:42.036640 20654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:42.036840 20654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:42.037552 20654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:42.038645 20654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:42:42.038750 20654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:42.038882 20654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:42:42.038968 20654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:42.051680 20654 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:32867
I20260809 04:42:42.052817 20654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:42:42.053697 20829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:32867 every 8 connection(s)
I20260809 04:42:42.060412 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 20654
I20260809 04:42:42.060936 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.222.212:59427/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:46637
--local_ip_for_outbound_sockets=127.5.222.194
--tserver_master_addrs=127.5.222.254:41287
--webserver_port=32873
--webserver_interface=127.5.222.194
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.222.212:59427/kms with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:42:42 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250561, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:42.071734 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:42.055093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:51683 (local address 127.5.222.254:41287)
0809 04:42:42.055652 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:42.055675 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:42.055970 (+ 295us) server_negotiation.cc:407] Connection header received
0809 04:42:42.057142 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:42.057202 (+ 60us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:42.057515 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:42.057829 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:42.058912 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.060019 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.061584 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.062042 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.062215 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:42.064959 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:42.064969 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:42.064987 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:42.065023 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:42.067305 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:42.068018 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:42.068027 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:42.068033 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:42.068187 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:42.068656 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:42.068662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:42.068664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:42.069231 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:42.069353 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:42.070194 (+ 841us) server_negotiation.cc:299] Negotiation successful
0809 04:42:42.070560 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":125,"threads_started":1}
I20260809 04:42:42.073935 20833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:42.055229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41287 (local address 127.5.222.193:51683)
0809 04:42:42.055778 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:42.055828 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:42.056692 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:42.057895 (+ 1203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:42.057903 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:42.058244 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:42.058742 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:42.058756 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.060151 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.060155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:42.061463 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:42.061473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.062204 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.062208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:42.062387 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:42.063081 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:42.063106 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:42.064731 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:42.067462 (+ 2731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:42.067471 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:42.067489 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:42.067860 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:42.068328 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:42.068334 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:42.068339 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:42.068530 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:42.069463 (+ 933us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:42.069471 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:42.069811 (+ 340us) client_negotiation.cc:769] Sending connection context
0809 04:42:42.072563 (+ 2752us) client_negotiation.cc:240] Negotiation successful
0809 04:42:42.072910 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":127,"threads_started":1}
I20260809 04:42:42.076960 20830 heartbeater.cc:347] Connected to a master server at 127.5.222.254:41287
I20260809 04:42:42.077225 20830 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:42.077819 20830 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
W20260809 04:42:42.080171 20830 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:41287 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 04:42:42.080309 20830 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:42.080485 20830 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
I20260809 04:42:42.081452 20830 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:42.081624 20830 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
W20260809 04:42:42.082458 20830 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:42:42.201120 20835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:42.201443 20835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:42.201511 20835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:42.205761 20835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:42.205840 20835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:42.205931 20835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:42:42.211336 20835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.222.212:59427/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:46637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=32873
--tserver_master_addrs=127.5.222.254:41287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:42.212432 20835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:42.213526 20835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:42.223724 20840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:42.224160 20843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:42.224273 20841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:42.224427 20835 server_base.cc:1034] running on GCE node
I20260809 04:42:42.224879 20835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:42.225667 20835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:42.228981 20835 hybrid_clock.cc:648] HybridClock initialized: now 1786250562228947 us; error 46 us; skew 500 ppm
Aug 09 04:42:42 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:42.231855 20835 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:42:42.232816 20835 webserver.cc:460] Webserver started at http://127.5.222.194:32873/ using document root <none> and password file <none>
I20260809 04:42:42.233434 20835 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:42.233511 20835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:42:42 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:42:42.256726 20835 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:42:42.260258 20853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:42.261832 20835 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:42:42.261989 20835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "5b6f3865a3c2400ab551c14af17bb59c"
format_stamp: "Formatted at 2026-08-09 04:42:40 on dist-test-slave-vxj2"
server_key: "22f24776dd98bae8623829365aea32fb"
server_key_iv: "6e853b728de217a0e739438724f985e4"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:42.262610 20835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:42.293951 20835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:42.298202 20835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:42.298481 20835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:42.299307 20835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:42.300536 20835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:42:42.300652 20835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:42.300788 20835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:42:42.300875 20835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:42.315955 20835 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:46637
I20260809 04:42:42.316004 20974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:46637 every 8 connection(s)
I20260809 04:42:42.317034 20835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:42:42.324944 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 20835
I20260809 04:42:42.325515 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.222.212:59427/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:46239
--local_ip_for_outbound_sockets=127.5.222.195
--tserver_master_addrs=127.5.222.254:41287
--webserver_port=37029
--webserver_interface=127.5.222.195
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.222.212:59427/kms with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:42:42 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:42.335095 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:42.319230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:40395 (local address 127.5.222.254:41287)
0809 04:42:42.319428 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:42.319433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:42.320278 (+ 845us) server_negotiation.cc:407] Connection header received
0809 04:42:42.321345 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:42.321349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:42.321404 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:42.321462 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:42.322763 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.323296 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.324617 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.325022 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.325144 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:42.330161 (+ 5017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:42.330168 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:42.330174 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:42.330209 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:42.332172 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:42.332878 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:42.332884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:42.332887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:42.332992 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:42.333529 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:42.333534 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:42.333536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:42.333799 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:42.333886 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:42.334679 (+ 793us) server_negotiation.cc:299] Negotiation successful
0809 04:42:42.334848 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 04:42:42.337231 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:42.319661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41287 (local address 127.5.222.194:40395)
0809 04:42:42.320124 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:42.320175 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:42.321051 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:42.321884 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:42.321890 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:42.322149 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:42.322613 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:42.322626 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.323423 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.323430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:42.324467 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:42.324477 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.327226 (+ 2749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.327232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:42.327471 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:42.328151 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:42.328180 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:42.329966 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:42.332336 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:42.332343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:42.332363 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:42.332744 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:42.333120 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:42.333125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:42.333128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:42.333355 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:42.333981 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:42.333989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:42.334358 (+ 369us) client_negotiation.cc:769] Sending connection context
0809 04:42:42.335610 (+ 1252us) client_negotiation.cc:240] Negotiation successful
0809 04:42:42.335983 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":142,"threads_started":1}
I20260809 04:42:42.338915 20975 heartbeater.cc:347] Connected to a master server at 127.5.222.254:41287
I20260809 04:42:42.339167 20975 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:42.339797 20975 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
W20260809 04:42:42.341526 20975 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:41287 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 04:42:42.341645 20975 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:42.341800 20975 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
I20260809 04:42:42.342689 20975 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:42.342845 20975 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
W20260809 04:42:42.343576 20975 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:42:42.466533 20978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:42.466890 20978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:42.466964 20978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:42.471611 20978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:42.471688 20978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:42.471800 20978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:42:42.477413 20978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.222.212:59427/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:46239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=37029
--tserver_master_addrs=127.5.222.254:41287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:42.478662 20978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:42.479780 20978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:42.489609 20988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:42.489596 20984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:42.490386 20985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:42.490962 20978 server_base.cc:1034] running on GCE node
I20260809 04:42:42.491540 20978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:42.492295 20978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:42.495579 20978 hybrid_clock.cc:648] HybridClock initialized: now 1786250562495542 us; error 46 us; skew 500 ppm
Aug 09 04:42:42 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:42.499166 20978 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:42:42.500249 20978 webserver.cc:460] Webserver started at http://127.5.222.195:37029/ using document root <none> and password file <none>
I20260809 04:42:42.500981 20978 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:42.501084 20978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:42:42 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:42:42.529314 20978 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.004s sys 0.000s
I20260809 04:42:42.532188 20997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:42.533794 20978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:42:42.534001 20978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "4985f51e10a0401a881ceb743244db01"
format_stamp: "Formatted at 2026-08-09 04:42:41 on dist-test-slave-vxj2"
server_key: "8e0299f1709e389b5822ac3cb5ebddf5"
server_key_iv: "06d1badac6a5d4f4e407d22d1aaa4530"
server_key_version: "kuduclusterkey@0"
I20260809 04:42:42.534804 20978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:42.571018 20978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:42.574333 20978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:42.574544 20978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:42.575212 20978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:42.576136 20978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:42:42.576207 20978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:42.576295 20978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:42:42.576341 20978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:42.589061 20978 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:46239
I20260809 04:42:42.590183 20978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:42:42.591097 21117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:46239 every 8 connection(s)
I20260809 04:42:42.595646 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 20978
Aug 09 04:42:42 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:42.607337 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:42.592292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:42419 (local address 127.5.222.254:41287)
0809 04:42:42.592427 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:42.592435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:42.593580 (+ 1145us) server_negotiation.cc:407] Connection header received
0809 04:42:42.594503 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:42.594507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:42.594565 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:42.594672 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:42.595988 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.596724 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.598323 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.598670 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.598757 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:42.602002 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:42.602008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:42.602013 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:42.602043 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:42.603739 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:42.604684 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:42.604943 (+ 259us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:42.604947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:42.605069 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:42.605558 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:42.605562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:42.605564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:42.605799 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:42.605866 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:42.606997 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0809 04:42:42.607116 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:42:42.608754 21120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:42.592856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41287 (local address 127.5.222.195:42419)
0809 04:42:42.593423 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:42.593466 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:42.594295 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:42.594854 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:42.594861 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:42.595122 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:42.595797 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:42.595819 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.596860 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.596862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:42.598153 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:42.598167 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.598966 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.598970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:42.599181 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:42.599867 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:42.599888 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:42.601793 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:42.603861 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:42.603868 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:42.603885 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:42.604554 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:42.605231 (+ 677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:42.605234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:42.605237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:42.605430 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:42.606335 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:42.606345 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:42.606704 (+ 359us) client_negotiation.cc:769] Sending connection context
0809 04:42:42.606942 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 04:42:42.607265 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":187,"threads_started":1}
I20260809 04:42:42.610213 21121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:42.596397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46950 (local address 127.5.222.254:41287)
0809 04:42:42.596622 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:42.596627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:42.597484 (+ 857us) server_negotiation.cc:407] Connection header received
0809 04:42:42.597662 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:42.597666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:42.597710 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:42.597781 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:42.598811 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.599547 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.600817 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:42.601047 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:42.601157 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:42.603897 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:42.603901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:42.603903 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:42.603927 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:42.605650 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:42.606532 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:42.606535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:42.606537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:42.606626 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:42.607772 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:42.607776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:42.607778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:42.607976 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:42.608166 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:42.609906 (+ 1740us) server_negotiation.cc:299] Negotiation successful
0809 04:42:42.610038 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":81,"threads_started":1}
I20260809 04:42:42.611285 21118 heartbeater.cc:347] Connected to a master server at 127.5.222.254:41287
I20260809 04:42:42.611533 21118 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:42.612126 21118 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
W20260809 04:42:42.613698 21118 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:41287 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 04:42:42.613842 21118 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:42.614029 21118 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
I20260809 04:42:42.614807 21118 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:42.614995 21118 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
W20260809 04:42:42.615746 21118 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 04:42:43 dist-test-slave-vxj2 krb5kdc[19511](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250563, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:43.082687 20830 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:43.083014 20830 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
I20260809 04:42:43.084183 20614 ts_manager.cc:194] Registered new tserver with Master: d46210f91c4c46a2b944e377fc024913 (127.5.222.193:32867)
I20260809 04:42:43.085881 20614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:51683
I20260809 04:42:43.343791 20975 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:43.344102 20975 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
I20260809 04:42:43.344870 20614 ts_manager.cc:194] Registered new tserver with Master: 5b6f3865a3c2400ab551c14af17bb59c (127.5.222.194:46637)
I20260809 04:42:43.345679 20614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:40395
Aug 09 04:42:43 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250563, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:42:43.616009 21118 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:43.616338 21118 heartbeater.cc:511] Master 127.5.222.254:41287 requested a full tablet report, sending...
I20260809 04:42:43.617273 20614 ts_manager.cc:194] Registered new tserver with Master: 4985f51e10a0401a881ceb743244db01 (127.5.222.195:46239)
I20260809 04:42:43.618078 20614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:42419
I20260809 04:42:43.623118 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:42:43.636736 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:43.624876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46952 (local address 127.5.222.254:41287)
0809 04:42:43.627317 (+ 2441us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:43.627321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:43.627334 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:42:43.627371 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:43.627374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:43.627422 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:43.627504 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:43.628434 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.629259 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.630555 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.630915 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.631021 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:43.632774 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:43.632778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:43.632781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:43.632810 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:43.634657 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:43.635162 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:43.635168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:43.635170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:43.635295 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:43.635630 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:43.635636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:43.635637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:43.635946 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:43.636107 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:43.636425 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 04:42:43.636558 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2354,"thread_start_us":95,"threads_started":1}
I20260809 04:42:43.641290 20614 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:42:43.646013 20614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:42:43.671115 21154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:43.660609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:32867 (local address 127.0.0.1:38734)
0809 04:42:43.663906 (+ 3297us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:43.663955 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:43.664095 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:43.665545 (+ 1450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:43.665551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:43.665573 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:43.665877 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:43.665885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.667331 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.667334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:43.668761 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:43.668773 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.670569 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.670573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:43.670690 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:43.670737 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 04:42:43.670831 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 04:42:43.670951 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3116,"mutex_wait_us":167,"thread_start_us":95,"threads_started":1}
I20260809 04:42:43.671854 21149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:43.662495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:46239 (local address 127.0.0.1:41810)
0809 04:42:43.664762 (+ 2267us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:43.664775 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:43.664867 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:43.665608 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:43.665612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:43.665633 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:43.665770 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:43.665777 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.668160 (+ 2383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.668164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:43.669425 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:43.669437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.671460 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.671464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:43.671554 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:43.671578 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:42:43.671653 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 04:42:43.671720 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2213,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260809 04:42:43.672179 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:43.661408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38734 (local address 127.5.222.193:32867)
0809 04:42:43.662546 (+ 1138us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:43.662554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:43.665071 (+ 2517us) server_negotiation.cc:407] Connection header received
0809 04:42:43.665180 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:43.665185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:43.665377 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:43.665708 (+ 331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:43.666004 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.667188 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.668926 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.669921 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.671750 (+ 1829us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:43.671891 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:43.671945 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 04:42:43.672036 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1015,"thread_start_us":65,"threads_started":1}
I20260809 04:42:43.672578 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:43.662663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41810 (local address 127.5.222.195:46239)
0809 04:42:43.664386 (+ 1723us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:43.664392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:43.664862 (+ 470us) server_negotiation.cc:407] Connection header received
0809 04:42:43.664978 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:43.664983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:43.665235 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:43.665418 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:43.665914 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.667924 (+ 2010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.669577 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.671100 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.671221 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:43.671393 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:43.672345 (+ 952us) server_negotiation.cc:299] Negotiation successful
0809 04:42:43.672443 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1620,"thread_start_us":97,"threads_started":1}
I20260809 04:42:43.675807 21050 tablet_service.cc:1467] Processing CreateTablet for tablet e384cb752a3040bb8047b612242d70fb (DEFAULT_TABLE table=test-table [id=c61c663c75ee448b9703fe45fdb64207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:42:43.675819 20757 tablet_service.cc:1467] Processing CreateTablet for tablet e384cb752a3040bb8047b612242d70fb (DEFAULT_TABLE table=test-table [id=c61c663c75ee448b9703fe45fdb64207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:42:43.676942 20757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e384cb752a3040bb8047b612242d70fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:43.676947 21050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e384cb752a3040bb8047b612242d70fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:43.686542 21157 tablet_bootstrap.cc:492] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01: Bootstrap starting.
I20260809 04:42:43.688261 21158 tablet_bootstrap.cc:492] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913: Bootstrap starting.
I20260809 04:42:43.690747 21158 tablet_bootstrap.cc:654] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:43.691392 21152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:43.663424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:46637 (local address 127.0.0.1:36132)
0809 04:42:43.670332 (+ 6908us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:43.670346 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:43.670438 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:43.686618 (+ 16180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:43.686622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:43.686644 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:43.686743 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:43.686750 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.688344 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.688348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:43.689643 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:43.689655 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.690950 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.690954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:43.691051 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:43.691076 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:42:43.691152 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 04:42:43.691228 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6847,"thread_start_us":81,"threads_started":1}
I20260809 04:42:43.692096 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:43.663805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36132 (local address 127.5.222.194:46637)
0809 04:42:43.670181 (+ 6376us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:43.670187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:43.672436 (+ 2249us) server_negotiation.cc:407] Connection header received
0809 04:42:43.672529 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:43.672535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:43.672854 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:43.686384 (+ 13530us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:43.686971 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.688176 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.689795 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:43.690631 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:43.690739 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:43.690883 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:43.691843 (+ 960us) server_negotiation.cc:299] Negotiation successful
0809 04:42:43.691948 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6265,"thread_start_us":78,"threads_started":1}
I20260809 04:42:43.692250 21158 log.cc:824] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:43.695259 20901 tablet_service.cc:1467] Processing CreateTablet for tablet e384cb752a3040bb8047b612242d70fb (DEFAULT_TABLE table=test-table [id=c61c663c75ee448b9703fe45fdb64207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:42:43.696408 20901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e384cb752a3040bb8047b612242d70fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:43.699421 21157 tablet_bootstrap.cc:654] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:43.700294 21157 log.cc:824] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:43.704898 21158 tablet_bootstrap.cc:492] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913: No bootstrap required, opened a new log
I20260809 04:42:43.705144 21158 ts_tablet_manager.cc:1397] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.000s
I20260809 04:42:43.709378 21158 raft_consensus.cc:348] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46210f91c4c46a2b944e377fc024913" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32867 } } peers { permanent_uuid: "4985f51e10a0401a881ceb743244db01" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46239 } } peers { permanent_uuid: "5b6f3865a3c2400ab551c14af17bb59c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46637 } }
Aug 09 04:42:43 dist-test-slave-vxj2 krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250563, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for HTTP/127.5.222.212@KRBTEST.COM
I20260809 04:42:43.709854 21158 raft_consensus.cc:374] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:43.710003 21158 raft_consensus.cc:729] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d46210f91c4c46a2b944e377fc024913, State: Initialized, Role: FOLLOWER
I20260809 04:42:43.710563 21158 consensus_queue.cc:260] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46210f91c4c46a2b944e377fc024913" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32867 } } peers { permanent_uuid: "4985f51e10a0401a881ceb743244db01" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46239 } } peers { permanent_uuid: "5b6f3865a3c2400ab551c14af17bb59c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46637 } }
I20260809 04:42:43.711737 21158 ts_tablet_manager.cc:1428] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:42:43.712168 20830 heartbeater.cc:503] Master 127.5.222.254:41287 was elected leader, sending a full tablet report...
I20260809 04:42:43.712296 21162 tablet_bootstrap.cc:492] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c: Bootstrap starting.
I20260809 04:42:43.714711 21162 tablet_bootstrap.cc:654] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:43.716867 21157 tablet_bootstrap.cc:492] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01: No bootstrap required, opened a new log
I20260809 04:42:43.717221 21157 ts_tablet_manager.cc:1397] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01: Time spent bootstrapping tablet: real 0.031s user 0.006s sys 0.000s
I20260809 04:42:43.718257 21162 log.cc:824] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:43.720883 21162 tablet_bootstrap.cc:492] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c: No bootstrap required, opened a new log
I20260809 04:42:43.721088 21162 ts_tablet_manager.cc:1397] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260809 04:42:43.724177 21157 raft_consensus.cc:348] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46210f91c4c46a2b944e377fc024913" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32867 } } peers { permanent_uuid: "4985f51e10a0401a881ceb743244db01" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46239 } } peers { permanent_uuid: "5b6f3865a3c2400ab551c14af17bb59c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46637 } }
I20260809 04:42:43.725131 21162 raft_consensus.cc:348] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46210f91c4c46a2b944e377fc024913" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32867 } } peers { permanent_uuid: "4985f51e10a0401a881ceb743244db01" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46239 } } peers { permanent_uuid: "5b6f3865a3c2400ab551c14af17bb59c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46637 } }
I20260809 04:42:43.725404 21157 raft_consensus.cc:374] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:43.725458 21162 raft_consensus.cc:374] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:43.725515 21162 raft_consensus.cc:729] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b6f3865a3c2400ab551c14af17bb59c, State: Initialized, Role: FOLLOWER
I20260809 04:42:43.725559 21157 raft_consensus.cc:729] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4985f51e10a0401a881ceb743244db01, State: Initialized, Role: FOLLOWER
I20260809 04:42:43.726011 21162 consensus_queue.cc:260] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46210f91c4c46a2b944e377fc024913" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32867 } } peers { permanent_uuid: "4985f51e10a0401a881ceb743244db01" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46239 } } peers { permanent_uuid: "5b6f3865a3c2400ab551c14af17bb59c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46637 } }
I20260809 04:42:43.726100 21157 consensus_queue.cc:260] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46210f91c4c46a2b944e377fc024913" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32867 } } peers { permanent_uuid: "4985f51e10a0401a881ceb743244db01" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46239 } } peers { permanent_uuid: "5b6f3865a3c2400ab551c14af17bb59c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46637 } }
I20260809 04:42:43.727137 21162 ts_tablet_manager.cc:1428] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260809 04:42:43.727209 21157 ts_tablet_manager.cc:1428] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260809 04:42:43.727563 20975 heartbeater.cc:503] Master 127.5.222.254:41287 was elected leader, sending a full tablet report...
I20260809 04:42:43.727607 21118 heartbeater.cc:503] Master 127.5.222.254:41287 was elected leader, sending a full tablet report...
W20260809 04:42:43.805536 20831 tablet.cc:2369] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:42:43.820093 20976 tablet.cc:2369] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:42:43.842088 21119 tablet.cc:2369] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:42:44.008496 21165 raft_consensus.cc:482] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:42:44.008793 21165 raft_consensus.cc:504] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46210f91c4c46a2b944e377fc024913" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32867 } } peers { permanent_uuid: "4985f51e10a0401a881ceb743244db01" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46239 } } peers { permanent_uuid: "5b6f3865a3c2400ab551c14af17bb59c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46637 } }
I20260809 04:42:44.010049 21165 leader_election.cc:288] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d46210f91c4c46a2b944e377fc024913 (127.5.222.193:32867), 4985f51e10a0401a881ceb743244db01 (127.5.222.195:46239)
I20260809 04:42:44.014818 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:44.010330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:32867 (local address 127.5.222.194:55641)
0809 04:42:44.010898 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:44.010913 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:44.011064 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:44.011392 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:44.011396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:44.011415 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:44.011563 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:44.011572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:44.012568 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:44.012571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:44.013700 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:44.013707 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:44.014342 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:44.014345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:44.014456 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:44.014478 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:42:44.014554 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 04:42:44.014629 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":256,"thread_start_us":125,"threads_started":1}
I20260809 04:42:44.014818 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:44.010514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:55641 (local address 127.5.222.193:32867)
0809 04:42:44.010638 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:44.010642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:44.010991 (+ 349us) server_negotiation.cc:407] Connection header received
0809 04:42:44.011199 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:44.011205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:44.011262 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:44.011330 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:44.011690 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:44.012439 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:44.013812 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:44.014212 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:44.014346 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:44.014401 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:44.014594 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 04:42:44.014681 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:42:44.014938 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:44.010472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:43031 (local address 127.5.222.195:46239)
0809 04:42:44.010619 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:44.010624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:44.010928 (+ 304us) server_negotiation.cc:407] Connection header received
0809 04:42:44.011089 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:44.011092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:44.011176 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:44.011253 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:44.011596 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:44.012135 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:44.013677 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:44.014266 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:44.014385 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:44.014432 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:44.014707 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 04:42:44.014788 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:42:44.015268 21176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:44.010329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:46239 (local address 127.5.222.194:43031)
0809 04:42:44.010796 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:44.010813 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:44.010966 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:44.011286 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:44.011289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:44.011305 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:44.011457 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:44.011463 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:44.012253 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:44.012257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:44.013539 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:44.013549 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:44.014473 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:44.014476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:44.014571 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:44.014588 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 04:42:44.015076 (+ 488us) client_negotiation.cc:240] Negotiation successful
0809 04:42:44.015142 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":106,"threads_started":1}
I20260809 04:42:44.015975 20764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e384cb752a3040bb8047b612242d70fb" candidate_uuid: "5b6f3865a3c2400ab551c14af17bb59c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d46210f91c4c46a2b944e377fc024913" is_pre_election: true
I20260809 04:42:44.016038 21067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e384cb752a3040bb8047b612242d70fb" candidate_uuid: "5b6f3865a3c2400ab551c14af17bb59c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4985f51e10a0401a881ceb743244db01" is_pre_election: true
I20260809 04:42:44.016315 20764 raft_consensus.cc:2454] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b6f3865a3c2400ab551c14af17bb59c in term 0.
I20260809 04:42:44.016325 21067 raft_consensus.cc:2454] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b6f3865a3c2400ab551c14af17bb59c in term 0.
I20260809 04:42:44.016786 20856 leader_election.cc:302] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4985f51e10a0401a881ceb743244db01, 5b6f3865a3c2400ab551c14af17bb59c; no voters:
I20260809 04:42:44.017093 21165 raft_consensus.cc:2781] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:42:44.017206 21165 raft_consensus.cc:482] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:42:44.017259 21165 raft_consensus.cc:3037] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:44.018383 21165 raft_consensus.cc:504] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46210f91c4c46a2b944e377fc024913" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32867 } } peers { permanent_uuid: "4985f51e10a0401a881ceb743244db01" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46239 } } peers { permanent_uuid: "5b6f3865a3c2400ab551c14af17bb59c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46637 } }
I20260809 04:42:44.018708 21165 leader_election.cc:288] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [CANDIDATE]: Term 1 election: Requested vote from peers d46210f91c4c46a2b944e377fc024913 (127.5.222.193:32867), 4985f51e10a0401a881ceb743244db01 (127.5.222.195:46239)
I20260809 04:42:44.019055 20764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e384cb752a3040bb8047b612242d70fb" candidate_uuid: "5b6f3865a3c2400ab551c14af17bb59c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d46210f91c4c46a2b944e377fc024913"
I20260809 04:42:44.019054 21067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e384cb752a3040bb8047b612242d70fb" candidate_uuid: "5b6f3865a3c2400ab551c14af17bb59c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4985f51e10a0401a881ceb743244db01"
I20260809 04:42:44.019184 21067 raft_consensus.cc:3037] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:44.019189 20764 raft_consensus.cc:3037] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:44.020246 20764 raft_consensus.cc:2454] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b6f3865a3c2400ab551c14af17bb59c in term 1.
I20260809 04:42:44.020532 21067 raft_consensus.cc:2454] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b6f3865a3c2400ab551c14af17bb59c in term 1.
I20260809 04:42:44.020546 20854 leader_election.cc:302] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b6f3865a3c2400ab551c14af17bb59c, d46210f91c4c46a2b944e377fc024913; no voters:
I20260809 04:42:44.020730 21165 raft_consensus.cc:2781] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:42:44.020944 21165 raft_consensus.cc:686] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 1 LEADER]: Becoming Leader. State: Replica: 5b6f3865a3c2400ab551c14af17bb59c, State: Running, Role: LEADER
I20260809 04:42:44.021291 21165 consensus_queue.cc:237] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46210f91c4c46a2b944e377fc024913" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32867 } } peers { permanent_uuid: "4985f51e10a0401a881ceb743244db01" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46239 } } peers { permanent_uuid: "5b6f3865a3c2400ab551c14af17bb59c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46637 } }
I20260809 04:42:44.024842 20614 catalog_manager.cc:5495] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b6f3865a3c2400ab551c14af17bb59c (127.5.222.194). New cstate: current_term: 1 leader_uuid: "5b6f3865a3c2400ab551c14af17bb59c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46210f91c4c46a2b944e377fc024913" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4985f51e10a0401a881ceb743244db01" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b6f3865a3c2400ab551c14af17bb59c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46637 } health_report { overall_health: HEALTHY } } }
I20260809 04:42:44.122856 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:44.118737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.5.222.194:46637)
0809 04:42:44.118896 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:44.118901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:44.118919 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:42:44.119036 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:44.119039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:44.119106 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:44.119221 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:42:44.119441 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:44.120286 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:44.121179 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:44.121516 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:44.121628 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:44.121893 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:42:44.122436 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:42:44.122561 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:44.122601 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 04:42:44.122660 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:42:44.129078 21067 raft_consensus.cc:1261] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01 [term 1 FOLLOWER]: Refusing update from remote peer 5b6f3865a3c2400ab551c14af17bb59c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:42:44.129078 20764 raft_consensus.cc:1261] T e384cb752a3040bb8047b612242d70fb P d46210f91c4c46a2b944e377fc024913 [term 1 FOLLOWER]: Refusing update from remote peer 5b6f3865a3c2400ab551c14af17bb59c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:42:44.129943 21178 consensus_queue.cc:1035] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d46210f91c4c46a2b944e377fc024913" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 32867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:42:44.130142 21165 consensus_queue.cc:1035] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4985f51e10a0401a881ceb743244db01" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 46239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:42:44.137516 21181 mvcc.cc:205] Tried to move back new op lower bound from 7316482310665142272 to 7316482310233780224. Current Snapshot: MvccSnapshot[applied={T|T < 7316482310665142272}]
I20260809 04:42:44.138420 21182 mvcc.cc:205] Tried to move back new op lower bound from 7316482310665142272 to 7316482310233780224. Current Snapshot: MvccSnapshot[applied={T|T < 7316482310665142272}]
I20260809 04:42:44.141755 21184 mvcc.cc:205] Tried to move back new op lower bound from 7316482310665142272 to 7316482310233780224. Current Snapshot: MvccSnapshot[applied={T|T < 7316482310665142272}]
I20260809 04:42:44.151283 20614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:42:44.151465 20614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:42:44.154142 20614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c25491653d94a82a3d77b155f24b847: Sending DeleteTablet for 3 replicas of tablet e384cb752a3040bb8047b612242d70fb
I20260809 04:42:44.154970 20757 tablet_service.cc:1514] Processing DeleteTablet for tablet e384cb752a3040bb8047b612242d70fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:42:44 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:38734
I20260809 04:42:44.155196 21050 tablet_service.cc:1514] Processing DeleteTablet for tablet e384cb752a3040bb8047b612242d70fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:42:44 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:41810
I20260809 04:42:44.155171 20901 tablet_service.cc:1514] Processing DeleteTablet for tablet e384cb752a3040bb8047b612242d70fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:42:44 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:36132
I20260809 04:42:44.155807 21195 tablet_replica.cc:330] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01: stopping tablet replica
I20260809 04:42:44.156004 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 20654
I20260809 04:42:44.156190 21195 raft_consensus.cc:2229] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:42:44.156702 21195 raft_consensus.cc:2258] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:42:44.163897 21192 tablet_replica.cc:330] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c: stopping tablet replica
I20260809 04:42:44.164315 21192 raft_consensus.cc:2229] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:42:44.164829 21192 raft_consensus.cc:2258] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:42:44.166610 21192 ts_tablet_manager.cc:1905] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:42:44.169906 21192 ts_tablet_manager.cc:1918] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260809 04:42:44.169907 20596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:32867 (error 108)
I20260809 04:42:44.170068 21192 log.cc:1196] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/e384cb752a3040bb8047b612242d70fb
I20260809 04:42:44.170404 21192 ts_tablet_manager.cc:1939] T e384cb752a3040bb8047b612242d70fb P 5b6f3865a3c2400ab551c14af17bb59c: Deleting consensus metadata
I20260809 04:42:44.170823 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 20835
I20260809 04:42:44.173930 21195 ts_tablet_manager.cc:1905] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:42:44.174218 21152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:44.173770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:32867 (local address 127.0.0.1:38746)
0809 04:42:44.173980 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:44.174064 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:32867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260809 04:42:44.174501 20596 catalog_manager.cc:4584] TS d46210f91c4c46a2b944e377fc024913 (127.5.222.193:32867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e384cb752a3040bb8047b612242d70fb: Network error: Client connection negotiation failed: client connection to 127.5.222.193:32867: connect: Connection refused (error 111)
I20260809 04:42:44.176960 21195 ts_tablet_manager.cc:1918] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:42:44.177073 21195 log.cc:1196] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/e384cb752a3040bb8047b612242d70fb
I20260809 04:42:44.177480 21195 ts_tablet_manager.cc:1939] T e384cb752a3040bb8047b612242d70fb P 4985f51e10a0401a881ceb743244db01: Deleting consensus metadata
I20260809 04:42:44.178730 20598 catalog_manager.cc:4841] TS 4985f51e10a0401a881ceb743244db01 (127.5.222.195:46239): tablet e384cb752a3040bb8047b612242d70fb (table test-table [id=c61c663c75ee448b9703fe45fdb64207]) successfully deleted
I20260809 04:42:44.179309 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 20978
I20260809 04:42:44.179337 21152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:44.179050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:46637 (local address 127.0.0.1:36148)
0809 04:42:44.179206 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:44.179262 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.194:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 04:42:44.187047 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 20576
I20260809 04:42:44.197830 6011 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 04:42:44.542557 6011 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 04:42:44.542716 6011 mini_ranger.cc:66] Stopping Ranger...
I20260809 04:42:44.887643 6011 mini_ranger.cc:68] Stopped Ranger
2026-08-09 04:42:44.887 UTC [19546] LOG: received smart shutdown request
2026-08-09 04:42:44.891 UTC [19546] LOG: background worker "logical replication launcher" (PID 19556) exited with exit code 1
2026-08-09 04:42:44.892 UTC [19550] LOG: shutting down
2026-08-09 04:42:45.050 UTC [19546] LOG: database system is shut down
2026-08-09T04:42:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50554 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:42:45 dist-test-slave-vxj2 krb5kdc[21206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:42:45 dist-test-slave-vxj2 krb5kdc[21206](info): set up 2 sockets
Aug 09 04:42:45 dist-test-slave-vxj2 krb5kdc[21206](info): commencing operation
krb5kdc: starting...
W20260809 04:42:46.288702 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:42:46 dist-test-slave-vxj2 krb5kdc[21206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:42:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:42:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:42:46.415692 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:35179
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:34509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:35179,127.5.222.253:38883,127.5.222.252:42603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:42:46.526413 21221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:46.526700 21221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:46.526742 21221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:46.530978 21221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:42:46.531075 21221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:46.531096 21221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:46.531114 21221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:42:46.531136 21221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:42:46.536979 21221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:34509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:35179,127.5.222.253:38883,127.5.222.252:42603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:35179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:46.538338 21221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:46.539479 21221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:46.548624 21230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:46.548600 21226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:46.548699 21221 server_base.cc:1034] running on GCE node
W20260809 04:42:46.548600 21227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:46.549679 21221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:46.550776 21221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:46.551952 21221 hybrid_clock.cc:648] HybridClock initialized: now 1786250566551919 us; error 45 us; skew 500 ppm
Aug 09 04:42:46 dist-test-slave-vxj2 krb5kdc[21206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250566, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:46.554929 21221 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:42:46.555835 21221 webserver.cc:460] Webserver started at http://127.5.222.254:46759/ using document root <none> and password file <none>
I20260809 04:42:46.556386 21221 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:46.556448 21221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:46.556682 21221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:42:46.558774 21221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "b9ba2cdac65343c684bccb6c4bce2aa1"
format_stamp: "Formatted at 2026-08-09 04:42:46 on dist-test-slave-vxj2"
server_key: "eb7a566c0260320a1721ec359365e32b"
server_key_iv: "69dc5da370ef7c7a00d6023f515da2b4"
server_key_version: "encryptionkey@0"
I20260809 04:42:46.559352 21221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "b9ba2cdac65343c684bccb6c4bce2aa1"
format_stamp: "Formatted at 2026-08-09 04:42:46 on dist-test-slave-vxj2"
server_key: "eb7a566c0260320a1721ec359365e32b"
server_key_iv: "69dc5da370ef7c7a00d6023f515da2b4"
server_key_version: "encryptionkey@0"
I20260809 04:42:46.563217 21221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:42:46.565651 21237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:46.567023 21221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:42:46.567195 21221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "b9ba2cdac65343c684bccb6c4bce2aa1"
format_stamp: "Formatted at 2026-08-09 04:42:46 on dist-test-slave-vxj2"
server_key: "eb7a566c0260320a1721ec359365e32b"
server_key_iv: "69dc5da370ef7c7a00d6023f515da2b4"
server_key_version: "encryptionkey@0"
I20260809 04:42:46.567322 21221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:46.591924 21221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:46.595783 21221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:46.596035 21221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:46.604130 21221 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:35179
I20260809 04:42:46.604164 21293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:35179 every 8 connection(s)
I20260809 04:42:46.605137 21221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:42:46.607867 21294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:46.611654 21294 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:42:46.612485 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 21221
I20260809 04:42:46.612649 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:42:46.612952 6011 external_mini_cluster.cc:1428] Setting key c1507c46284a18203d0bc61fb94fc901
WARNING: no policy specified for kudu/127.5.222.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.253@KRBTEST.COM" created.
Aug 09 04:42:46 dist-test-slave-vxj2 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250566, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:46.631522 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.614105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35434 (local address 127.5.222.254:35179)
0809 04:42:46.614654 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:46.614675 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:46.614930 (+ 255us) server_negotiation.cc:407] Connection header received
0809 04:42:46.615956 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:46.615964 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:46.616155 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:46.616353 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:46.617515 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.618486 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.620089 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.620462 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.620576 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:46.626285 (+ 5709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:46.626290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:46.626302 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:46.626325 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:46.628113 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:46.628678 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:46.628684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:46.628688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:46.628803 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:46.629139 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:46.629144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:46.629146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:46.629559 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:46.629716 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:46.630106 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 04:42:46.630449 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":364,"thread_start_us":139,"threads_started":1}
I20260809 04:42:46.631518 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.614105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:35179 (local address 127.0.0.1:35434)
0809 04:42:46.614757 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.614806 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:46.615634 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:46.616423 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:46.616431 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:46.616920 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:46.617344 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:46.617367 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.618639 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.618645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:46.619947 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:46.619963 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.620639 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.620646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:46.620788 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:46.621491 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:46.621517 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:46.626108 (+ 4591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:46.628276 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:46.628285 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:46.628301 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:46.628559 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:46.628906 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:46.628911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:46.628912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:46.629026 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:46.629688 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:46.629695 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:46.629859 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 04:42:46.630031 (+ 172us) client_negotiation.cc:240] Negotiation successful
0809 04:42:46.630501 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":138,"spinlock_wait_cycles":30592,"thread_start_us":171,"threads_started":1}
I20260809 04:42:46.633891 21294 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:42:46.634789 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.634483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:38883 (local address 127.0.0.1:43110)
0809 04:42:46.634601 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.634719 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:38883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 04:42:46.634989 21240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.222.253:38883: connect: Connection refused (error 111)
I20260809 04:42:46.635892 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.635677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:38883 (local address 127.0.0.1:43116)
0809 04:42:46.635788 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.635839 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:38883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 04:42:46.636068 21294 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.253:38883: Network error: Client connection negotiation failed: client connection to 127.5.222.253:38883: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.253.keytab.
Entry for principal kudu/127.5.222.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.253.keytab.
I20260809 04:42:46.640167 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.253:38883
--webserver_interface=127.5.222.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.253.keytab
--principal=kudu/127.5.222.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:34509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:35179,127.5.222.253:38883,127.5.222.252:42603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:42:46.685281 21294 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } attempt: 1
I20260809 04:42:46.686218 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.685900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:38883 (local address 127.0.0.1:43122)
0809 04:42:46.686064 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.686136 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:38883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 04:42:46.687214 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.686862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:38883 (local address 127.0.0.1:43124)
0809 04:42:46.687057 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.687126 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:38883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260809 04:42:46.687433 21294 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.253:38883: Network error: Client connection negotiation failed: client connection to 127.5.222.253:38883: connect: Connection refused (error 111)
W20260809 04:42:46.754786 21302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:46.755048 21302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:46.755089 21302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 04:42:46.755616 21294 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } attempt: 2
I20260809 04:42:46.756482 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.756178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:38883 (local address 127.0.0.1:43132)
0809 04:42:46.756332 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.756394 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:38883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 04:42:46.757279 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.757011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:38883 (local address 127.0.0.1:43138)
0809 04:42:46.757121 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.757224 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:38883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 04:42:46.757391 21294 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.253:38883: Network error: Client connection negotiation failed: client connection to 127.5.222.253:38883: connect: Connection refused (error 111)
W20260809 04:42:46.759495 21302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:42:46.759580 21302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:46.759634 21302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:46.759696 21302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:42:46.759740 21302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:42:46.764928 21302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:34509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:35179,127.5.222.253:38883,127.5.222.252:42603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.253.keytab
--principal=kudu/127.5.222.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.253:38883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.222.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:46.765972 21302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:46.767000 21302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:46.775599 21312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:46.775606 21309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:46.775812 21302 server_base.cc:1034] running on GCE node
W20260809 04:42:46.775674 21310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:46.776484 21302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:46.777667 21302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:46.778889 21302 hybrid_clock.cc:648] HybridClock initialized: now 1786250566778872 us; error 26 us; skew 500 ppm
Aug 09 04:42:46 dist-test-slave-vxj2 krb5kdc[21206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250566, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:46.781827 21302 init.cc:377] Logged in from keytab as kudu/127.5.222.253@KRBTEST.COM (short username kudu)
I20260809 04:42:46.782670 21302 webserver.cc:460] Webserver started at http://127.5.222.253:39165/ using document root <none> and password file <none>
I20260809 04:42:46.783209 21302 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:46.783303 21302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:46.783526 21302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:42:46.785544 21302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/data/instance:
uuid: "58e1d241e53f46cc884fa7241cc75f03"
format_stamp: "Formatted at 2026-08-09 04:42:46 on dist-test-slave-vxj2"
server_key: "35684d7aedaa681ee0e1e243d61ce7c6"
server_key_iv: "b407aafb0b37464c4515f10d26ba7aa5"
server_key_version: "encryptionkey@0"
I20260809 04:42:46.786098 21302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/wal/instance:
uuid: "58e1d241e53f46cc884fa7241cc75f03"
format_stamp: "Formatted at 2026-08-09 04:42:46 on dist-test-slave-vxj2"
server_key: "35684d7aedaa681ee0e1e243d61ce7c6"
server_key_iv: "b407aafb0b37464c4515f10d26ba7aa5"
server_key_version: "encryptionkey@0"
I20260809 04:42:46.790022 21302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:42:46.792358 21321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:46.793663 21302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:42:46.793820 21302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/wal
uuid: "58e1d241e53f46cc884fa7241cc75f03"
format_stamp: "Formatted at 2026-08-09 04:42:46 on dist-test-slave-vxj2"
server_key: "35684d7aedaa681ee0e1e243d61ce7c6"
server_key_iv: "b407aafb0b37464c4515f10d26ba7aa5"
server_key_version: "encryptionkey@0"
I20260809 04:42:46.793941 21302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:46.816437 21302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:46.819656 21302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:46.819907 21302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:46.827795 21302 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.253:38883
I20260809 04:42:46.827795 21376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.253:38883 every 8 connection(s)
I20260809 04:42:46.828801 21302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
I20260809 04:42:46.831527 21377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:46.835220 21377 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:42:46.836714 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 21302
I20260809 04:42:46.836846 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-1/wal/instance
I20260809 04:42:46.837251 6011 external_mini_cluster.cc:1428] Setting key 1f426750c7804234cacbc869fc36cdec
Aug 09 04:42:46 dist-test-slave-vxj2 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250566, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.253@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.252@KRBTEST.COM; defaulting to no policy
I20260809 04:42:46.848608 21294 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.252@KRBTEST.COM" created.
I20260809 04:42:46.850994 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.837252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35438 (local address 127.5.222.254:35179)
0809 04:42:46.837422 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:46.837427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:46.838533 (+ 1106us) server_negotiation.cc:407] Connection header received
0809 04:42:46.839725 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:46.839730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:46.839860 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:46.839977 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:46.841143 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.842203 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.843538 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.844058 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.844204 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:46.846942 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:46.846946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:46.846948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:46.846975 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:46.848467 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:46.849080 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:46.849084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:46.849086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:46.849213 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:46.849729 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:46.849733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:46.849734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:46.849983 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:46.850059 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:46.850693 (+ 634us) server_negotiation.cc:299] Negotiation successful
0809 04:42:46.850822 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:42:46.852172 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.837652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:35179 (local address 127.0.0.1:35438)
0809 04:42:46.838326 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.838391 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:46.839495 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:46.840146 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:46.840153 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:46.840485 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:46.840969 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:46.840983 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.842349 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.842352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:46.843396 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:46.843406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.844223 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.844225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:46.844366 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:46.844981 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:46.844997 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:46.846764 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:46.848622 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:46.848627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:46.848638 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:46.848950 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:46.849360 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:46.849367 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:46.849372 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:46.849567 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:46.850144 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:46.850152 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:46.850437 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 04:42:46.850631 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 04:42:46.850955 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":134,"threads_started":1}
I20260809 04:42:46.854025 21377 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:42:46 dist-test-slave-vxj2 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250566, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.253@KRBTEST.COM
I20260809 04:42:46.860389 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.849082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:38883 (local address 127.0.0.1:43148)
0809 04:42:46.849333 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.849351 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:46.849429 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:46.850255 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:46.850262 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:46.850474 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:46.851237 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:46.851262 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.852610 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.852615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:46.853347 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:46.853358 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.854002 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.854009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:46.854153 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:46.854661 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:46.854684 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:46.856257 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:46.858474 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:46.858477 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:46.858480 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:46.858683 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:46.858985 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:46.858989 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:46.858991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:46.859103 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:46.859739 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:46.859750 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:46.859874 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 04:42:46.860005 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 04:42:46.860173 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260809 04:42:46.860569 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.849451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43148 (local address 127.5.222.253:38883)
0809 04:42:46.849789 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:46.849803 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:46.849826 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:42:46.849867 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:46.849872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:46.850078 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:46.850219 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:46.851457 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.852490 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.853449 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.853797 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.853961 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:46.856384 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:46.856394 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:46.856418 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:46.856438 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:46.858327 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:46.858798 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:46.858804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:46.858808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:46.858889 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:46.859192 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:46.859195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:46.859198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:46.859622 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:46.859780 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:46.860129 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 04:42:46.860333 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":233,"thread_start_us":115,"threads_started":1}
Aug 09 04:42:46 dist-test-slave-vxj2 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250566, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.253@KRBTEST.COM for kudu/127.5.222.253@KRBTEST.COM
I20260809 04:42:46.862854 21294 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:42:46.863758 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.863375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50678)
0809 04:42:46.863607 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.863664 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260809 04:42:46.864567 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.864231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50692)
0809 04:42:46.864414 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.864497 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260809 04:42:46.864709 21294 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:42603: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
I20260809 04:42:46.866551 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.854573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:38883 (local address 127.0.0.1:43162)
0809 04:42:46.854734 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.854745 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:46.854800 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:46.856107 (+ 1307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:46.856113 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:46.856343 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:46.856505 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:46.856517 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.857977 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.857981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:46.858986 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:46.859004 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.859512 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.859519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:46.859681 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:46.860127 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:46.860141 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:46.862599 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:46.864900 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:46.864904 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:46.864908 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:46.865236 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:46.865502 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:46.865504 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:46.865505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:46.865587 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:46.866004 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:46.866006 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:46.866087 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 04:42:46.866180 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 04:42:46.866378 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":68}
I20260809 04:42:46.866566 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.855449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43162 (local address 127.5.222.253:38883)
0809 04:42:46.855862 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:46.855871 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:46.855888 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:42:46.855924 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:46.855930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:46.855978 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:46.856068 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:46.856932 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.857834 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.859135 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:46.859393 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:46.859507 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:46.862755 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:46.862759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:46.862761 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:46.862784 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:46.864805 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:46.865328 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:46.865331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:46.865332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:46.865404 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:46.865715 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:46.865721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:46.865723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:46.865910 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:46.865993 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:46.866256 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 04:42:46.866431 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":340,"thread_start_us":188,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.252.keytab.
Entry for principal kudu/127.5.222.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.252.keytab.
I20260809 04:42:46.867377 21377 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:42:46.868078 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.867824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50700)
0809 04:42:46.867945 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.868027 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 04:42:46.868283 21324 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
I20260809 04:42:46.868314 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.252:42603
--webserver_interface=127.5.222.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.252.keytab
--principal=kudu/127.5.222.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:34509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:35179,127.5.222.253:38883,127.5.222.252:42603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:42:46.869252 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.868975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50702)
0809 04:42:46.869089 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.869138 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 04:42:46.869423 21377 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:42603: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
I20260809 04:42:46.895961 21294 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } attempt: 1
I20260809 04:42:46.896898 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.896592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50718)
0809 04:42:46.896752 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.896817 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 04:42:46.897738 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.897549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50734)
0809 04:42:46.897648 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.897692 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 04:42:46.897891 21294 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:42603: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
I20260809 04:42:46.918746 21377 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } attempt: 1
I20260809 04:42:46.919636 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.919284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50742)
0809 04:42:46.919460 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.919540 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 04:42:46.920415 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.920172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50744)
0809 04:42:46.920292 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.920349 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 04:42:46.920564 21377 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:42603: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
I20260809 04:42:46.973085 21294 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } attempt: 2
I20260809 04:42:46.973977 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.973664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50746)
0809 04:42:46.973816 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.973881 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 04:42:46.974740 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.974546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50762)
0809 04:42:46.974647 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.974690 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 04:42:46.974949 21294 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:42603: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
W20260809 04:42:46.983997 21387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:46.984236 21387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:46.984319 21387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:46.988436 21387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:42:46.988490 21387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:46.988530 21387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:46.988565 21387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:42:46.988602 21387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:42:46.988747 21377 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } attempt: 2
I20260809 04:42:46.989575 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.989272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50766)
0809 04:42:46.989425 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.989492 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 04:42:46.990312 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:46.990086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50770)
0809 04:42:46.990192 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:46.990251 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 04:42:46.990440 21377 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:42603: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42603: connect: Connection refused (error 111)
I20260809 04:42:46.994338 21387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:34509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:35179,127.5.222.253:38883,127.5.222.252:42603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.252.keytab
--principal=kudu/127.5.222.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.252:42603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.222.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:46.995296 21387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:46.996351 21387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:47.004789 21393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:47.004899 21392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:47.004981 21387 server_base.cc:1034] running on GCE node
W20260809 04:42:47.005174 21395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:47.005717 21387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:47.006844 21387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:47.008018 21387 hybrid_clock.cc:648] HybridClock initialized: now 1786250567008003 us; error 25 us; skew 500 ppm
Aug 09 04:42:47 dist-test-slave-vxj2 krb5kdc[21206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250567, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:47.010998 21387 init.cc:377] Logged in from keytab as kudu/127.5.222.252@KRBTEST.COM (short username kudu)
I20260809 04:42:47.011860 21387 webserver.cc:460] Webserver started at http://127.5.222.252:38993/ using document root <none> and password file <none>
I20260809 04:42:47.012416 21387 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:47.012478 21387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:47.012707 21387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:42:47.014838 21387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/data/instance:
uuid: "d63260a3f97a487fb235be57740c257f"
format_stamp: "Formatted at 2026-08-09 04:42:47 on dist-test-slave-vxj2"
server_key: "6289eb00927cde3ebc4495e43ce12609"
server_key_iv: "953e77fe5bea3c66d69671e8788d1a61"
server_key_version: "encryptionkey@0"
I20260809 04:42:47.015417 21387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/wal/instance:
uuid: "d63260a3f97a487fb235be57740c257f"
format_stamp: "Formatted at 2026-08-09 04:42:47 on dist-test-slave-vxj2"
server_key: "6289eb00927cde3ebc4495e43ce12609"
server_key_iv: "953e77fe5bea3c66d69671e8788d1a61"
server_key_version: "encryptionkey@0"
I20260809 04:42:47.019134 21387 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:42:47.021502 21404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:47.022675 21387 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:42:47.022859 21387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/wal
uuid: "d63260a3f97a487fb235be57740c257f"
format_stamp: "Formatted at 2026-08-09 04:42:47 on dist-test-slave-vxj2"
server_key: "6289eb00927cde3ebc4495e43ce12609"
server_key_iv: "953e77fe5bea3c66d69671e8788d1a61"
server_key_version: "encryptionkey@0"
I20260809 04:42:47.022971 21387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:47.056623 21387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:47.060045 21387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:47.060263 21387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:47.067471 21387 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.252:42603
I20260809 04:42:47.067495 21459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.252:42603 every 8 connection(s)
I20260809 04:42:47.068389 21387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
I20260809 04:42:47.071013 21460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:47.074146 21294 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } attempt: 3
I20260809 04:42:47.075294 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 21387
I20260809 04:42:47.075066 21460 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:42:47.075448 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250403492198-6011-0/minicluster-data/master-2/wal/instance
I20260809 04:42:47.075800 6011 external_mini_cluster.cc:1428] Setting key 48a3c12ab856f414966ebfce16cb0c23
I20260809 04:42:47.081660 21377 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } attempt: 3
Aug 09 04:42:47 dist-test-slave-vxj2 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250566, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.252@KRBTEST.COM
Aug 09 04:42:47 dist-test-slave-vxj2 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250567, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.252@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
Aug 09 04:42:47 dist-test-slave-vxj2 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.252@KRBTEST.COM
Aug 09 04:42:47 dist-test-slave-vxj2 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250566, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.253@KRBTEST.COM for kudu/127.5.222.252@KRBTEST.COM
I20260809 04:42:47.089679 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.074723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50786)
0809 04:42:47.074905 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:47.074924 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:47.075025 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:47.077621 (+ 2596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:47.077624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:47.077834 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:47.077926 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:47.077933 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.079423 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.079426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:47.080091 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:47.080099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.081017 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.081021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:47.081121 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.081700 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:47.081717 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:47.083315 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:47.087133 (+ 3818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:47.087136 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:47.087138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:47.087527 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:47.088666 (+ 1139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:47.088671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:47.088673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:47.088760 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:47.089207 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:47.089210 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:47.089310 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 04:42:47.089393 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 04:42:47.089519 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":48}
I20260809 04:42:47.090709 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.076427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.5.222.252:42603)
0809 04:42:47.076715 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:47.076723 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:47.077031 (+ 308us) server_negotiation.cc:407] Connection header received
0809 04:42:47.077073 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:47.077076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:47.077397 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:47.077666 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:47.078472 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.079399 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.080576 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.080994 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.081071 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.084876 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:47.084881 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:47.084885 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:47.084914 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:47.087508 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.087969 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.087975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.087977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.088078 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.088362 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.088367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.088368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.088829 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:47.089044 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:47.089743 (+ 699us) server_negotiation.cc:299] Negotiation successful
0809 04:42:47.089890 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":202,"server-negotiator.queue_time_us":218,"spinlock_wait_cycles":5504,"thread_start_us":81,"threads_started":1}
I20260809 04:42:47.090708 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.075516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50786 (local address 127.5.222.252:42603)
0809 04:42:47.075997 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:47.076018 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:47.076060 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:42:47.076935 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:47.076960 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:47.077397 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:47.078027 (+ 630us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:47.078134 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.079294 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.080216 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.080905 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.081067 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.083488 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:47.083497 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:47.083512 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:47.083539 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:47.087020 (+ 3481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.088473 (+ 1453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.088476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.088478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.088554 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.088876 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.088879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.088880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.089063 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:47.089968 (+ 905us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:47.090048 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 04:42:47.090172 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":279,"spinlock_wait_cycles":66432,"thread_start_us":121,"threads_started":1}
I20260809 04:42:47.090838 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.076892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35452 (local address 127.5.222.254:35179)
0809 04:42:47.076998 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:47.077002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:47.077398 (+ 396us) server_negotiation.cc:407] Connection header received
0809 04:42:47.077607 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:47.077610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:47.077656 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:47.077711 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:47.078703 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.079561 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.080821 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.081125 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.081249 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.084384 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:47.084388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:47.084393 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:47.084420 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:47.086292 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.087119 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.087124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.087125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.087222 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.088213 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.088217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.088219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.088554 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:47.088652 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:47.090298 (+ 1646us) server_negotiation.cc:299] Negotiation successful
0809 04:42:47.090653 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":47}
I20260809 04:42:47.091646 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.076845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:35179 (local address 127.0.0.1:35452)
0809 04:42:47.077271 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:47.077318 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:47.077468 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:47.077922 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:47.077927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:47.078346 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:47.078550 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:47.078563 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.079677 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.079680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:47.080685 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:47.080695 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.081620 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.081624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:47.081734 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.082244 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:47.082474 (+ 230us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:47.084261 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:47.086448 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:47.086457 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:47.086465 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:47.087002 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:47.087920 (+ 918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:47.087923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:47.087925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:47.088096 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:47.088714 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:47.088722 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:47.088885 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 04:42:47.089183 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 04:42:47.089488 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":42,"spinlock_wait_cycles":2944,"thread_start_us":159,"threads_started":1}
I20260809 04:42:47.092634 21460 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:42:47.093881 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.082106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50804)
0809 04:42:47.082401 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:47.082417 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:47.082504 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:47.083557 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:47.083562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:47.083762 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:47.083880 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:47.083887 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.085889 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.085894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:47.086582 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:47.086596 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.087262 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.087276 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:47.087384 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.087888 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:47.087909 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:47.089351 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:47.091344 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:47.091347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:47.091349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:47.091543 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:47.092244 (+ 701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:47.092247 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:47.092251 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:47.092350 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:47.093381 (+ 1031us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:47.093384 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:47.093496 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 04:42:47.093585 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 04:42:47.093722 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260809 04:42:47.094466 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.082339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50804 (local address 127.5.222.252:42603)
0809 04:42:47.083294 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:47.083305 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:47.083320 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:42:47.083353 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:47.083359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:47.083405 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:47.084005 (+ 600us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:47.084107 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.085750 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.086718 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.087102 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.087254 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.089508 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:47.089511 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:47.089514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:47.089537 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:47.091228 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.091642 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.091647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.091649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.092130 (+ 481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.092670 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.092673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.092674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.092853 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:47.094063 (+ 1210us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:47.094137 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 04:42:47.094270 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":890,"thread_start_us":83,"threads_started":1}
I20260809 04:42:47.095342 21294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1: Bootstrap starting.
Aug 09 04:42:47 dist-test-slave-vxj2 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250567, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.252@KRBTEST.COM for kudu/127.5.222.253@KRBTEST.COM
I20260809 04:42:47.098547 21294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:47.099503 21294 log.cc:824] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:42:47 dist-test-slave-vxj2 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:47.102118 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.093185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.5.222.253:38883)
0809 04:42:47.093282 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:47.093286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:47.093469 (+ 183us) server_negotiation.cc:407] Connection header received
0809 04:42:47.093603 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:47.093606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:47.093646 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:47.093718 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:47.094466 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.095035 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.095975 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.096222 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.096329 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.098300 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:47.098304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:47.098306 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:47.098335 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:47.100220 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.100696 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.100699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.100701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.100788 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.101132 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.101136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.101138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.101434 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:47.101508 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:47.101853 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 04:42:47.101984 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 04:42:47.102197 21377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03: Bootstrap starting.
I20260809 04:42:47.102557 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.093082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:38883 (local address 127.0.0.1:43168)
0809 04:42:47.093399 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:47.093412 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:47.093526 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:47.093814 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:47.093817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:47.094250 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:47.094329 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:47.094336 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.095280 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.095283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:47.095852 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:47.095861 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.096330 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.096333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:47.096415 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.096829 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:47.096839 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:47.098162 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:47.100347 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:47.100351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:47.100353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:47.100599 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:47.100899 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:47.100902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:47.100904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:47.101013 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:47.101584 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:47.101587 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:47.101672 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 04:42:47.102306 (+ 634us) client_negotiation.cc:240] Negotiation successful
0809 04:42:47.102428 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248}
I20260809 04:42:47.102823 21294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1: No bootstrap required, opened a new log
I20260809 04:42:47.103170 21460 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:42:47.104431 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.093397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35456 (local address 127.5.222.254:35179)
0809 04:42:47.093498 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:47.093502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:47.093734 (+ 232us) server_negotiation.cc:407] Connection header received
0809 04:42:47.093900 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:47.093903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:47.093941 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:47.094003 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:47.094818 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.095471 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.096862 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.097197 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.097271 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.100352 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:47.100357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:47.100360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:47.100383 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:47.102030 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.102552 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.102555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.102557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.102646 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.102973 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.102975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.102977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.103170 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:47.104075 (+ 905us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:47.104146 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 04:42:47.104253 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 04:42:47.104625 21377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:47.105511 21377 log.cc:824] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:47.107336 21294 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } }
I20260809 04:42:47.107566 21294 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:47.107623 21294 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ba2cdac65343c684bccb6c4bce2aa1, State: Initialized, Role: FOLLOWER
I20260809 04:42:47.108105 21294 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } }
I20260809 04:42:47.108731 21377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03: No bootstrap required, opened a new log
Aug 09 04:42:47 dist-test-slave-vxj2 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250567, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.252@KRBTEST.COM for kudu/127.5.222.252@KRBTEST.COM
I20260809 04:42:47.109437 21294 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 04:42:47 dist-test-slave-vxj2 krb5kdc[21206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.253@KRBTEST.COM
I20260809 04:42:47.113265 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.103605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42603 (local address 127.0.0.1:50820)
0809 04:42:47.103870 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:47.103884 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:47.103965 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:47.104839 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:47.104841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:47.104998 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:47.105081 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:47.105087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.106125 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.106131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:47.106870 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:47.106880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.107438 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.107441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:47.107511 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.108133 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:47.108147 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:47.109808 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:47.111764 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:47.111767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:47.111771 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:47.112023 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:47.112305 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:47.112307 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:47.112309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:47.112379 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:47.112788 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:47.112791 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:47.112866 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 04:42:47.112960 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 04:42:47.113079 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":101,"threads_started":1}
I20260809 04:42:47.114076 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.103921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50820 (local address 127.5.222.252:42603)
0809 04:42:47.104613 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:47.104618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:47.104631 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:42:47.104663 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:47.104666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:47.104705 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:47.105204 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:47.105287 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.106005 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.106990 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.107323 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.107881 (+ 558us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.109960 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:47.109964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:47.109966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:47.109989 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:47.111589 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.112132 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.112134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.112136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.112215 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.112467 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.112469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.112470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.112679 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:47.113733 (+ 1054us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:47.113815 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 04:42:47.113943 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626}
I20260809 04:42:47.115043 21377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } }
I20260809 04:42:47.115265 21377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:47.115316 21377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58e1d241e53f46cc884fa7241cc75f03, State: Initialized, Role: FOLLOWER
I20260809 04:42:47.115806 21377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } }
I20260809 04:42:47.117031 21377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:42:47.117534 21460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f: Bootstrap starting.
I20260809 04:42:47.117795 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.105671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43172 (local address 127.5.222.253:38883)
0809 04:42:47.105767 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:47.105772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:47.106165 (+ 393us) server_negotiation.cc:407] Connection header received
0809 04:42:47.106275 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:47.106278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:47.106321 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:47.106390 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:47.107217 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.107908 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.108914 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.109244 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.109349 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.112259 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:47.112266 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:47.112280 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:47.112304 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:47.114622 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.115435 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.115438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.115440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.115525 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.115805 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.115808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.115809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.116007 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:47.116768 (+ 761us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:47.117510 (+ 742us) server_negotiation.cc:299] Negotiation successful
0809 04:42:47.117652 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 04:42:47.114713 21221 master_runner.cc:186] Error getting master registration for 127.5.222.253:38883: OK, instance_id { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" instance_seqno: 1786250566825416 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:42:47.118986 21221 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:42:47.120218 21460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:47.120700 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 04:42:47.120719 21485 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:42:47.120783 21485 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:42:47.120985 21470 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } } }
I20260809 04:42:47.121073 21470 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:42:47.121047 21460 log.cc:824] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:47.127038 21460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f: No bootstrap required, opened a new log
I20260809 04:42:47.124432 21302 master_runner.cc:186] Error getting master registration for 127.5.222.252:42603: OK, instance_id { permanent_uuid: "d63260a3f97a487fb235be57740c257f" instance_seqno: 1786250567065451 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:42:47.127674 21302 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:42:47.127838 21488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:42:47.127950 21488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } }
I20260809 04:42:47.129241 21267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58e1d241e53f46cc884fa7241cc75f03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" is_pre_election: true
I20260809 04:42:47.129524 21267 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58e1d241e53f46cc884fa7241cc75f03 in term 0.
I20260809 04:42:47.129906 21325 leader_election.cc:302] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58e1d241e53f46cc884fa7241cc75f03, b9ba2cdac65343c684bccb6c4bce2aa1; no voters:
W20260809 04:42:47.131636 21499 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:42:47.131732 21499 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:42:47.132257 21488 leader_election.cc:288] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9ba2cdac65343c684bccb6c4bce2aa1 (127.5.222.254:35179), d63260a3f97a487fb235be57740c257f (127.5.222.252:42603)
I20260809 04:42:47.132251 21460 raft_consensus.cc:348] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } }
I20260809 04:42:47.132463 21488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } } }
I20260809 04:42:47.132606 21460 raft_consensus.cc:374] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:47.132650 21488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:42:47.132754 21460 raft_consensus.cc:729] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63260a3f97a487fb235be57740c257f, State: Initialized, Role: FOLLOWER
I20260809 04:42:47.132792 21488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:42:47.132922 21488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:42:47.133033 21488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:47.133445 21460 consensus_queue.cc:260] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } }
I20260809 04:42:47.134743 21460 sys_catalog.cc:564] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:42:47.135504 21488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } }
I20260809 04:42:47.134805 21432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58e1d241e53f46cc884fa7241cc75f03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63260a3f97a487fb235be57740c257f" is_pre_election: true
I20260809 04:42:47.135867 21432 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58e1d241e53f46cc884fa7241cc75f03 in term 0.
I20260809 04:42:47.136368 21267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58e1d241e53f46cc884fa7241cc75f03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1"
I20260809 04:42:47.136511 21267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:47.137970 21267 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58e1d241e53f46cc884fa7241cc75f03 in term 1.
I20260809 04:42:47.138584 21325 leader_election.cc:302] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58e1d241e53f46cc884fa7241cc75f03, b9ba2cdac65343c684bccb6c4bce2aa1; no voters:
I20260809 04:42:47.142194 21509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:42:47.142310 21509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [term 1 LEADER]: Becoming Leader. State: Replica: 58e1d241e53f46cc884fa7241cc75f03, State: Running, Role: LEADER
I20260809 04:42:47.142613 21509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } }
I20260809 04:42:47.143672 21432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58e1d241e53f46cc884fa7241cc75f03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63260a3f97a487fb235be57740c257f"
I20260809 04:42:47.143790 21432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [term 0 FOLLOWER]: Advancing to term 1
W20260809 04:42:47.144554 21521 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:42:47.144675 21521 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:42:47.145023 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.126040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43176 (local address 127.5.222.253:38883)
0809 04:42:47.126135 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:47.126139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:47.127654 (+ 1515us) server_negotiation.cc:407] Connection header received
0809 04:42:47.128038 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:47.128041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:47.128090 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:47.128141 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:47.130207 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.130996 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.134222 (+ 3226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.134556 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.134631 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.138039 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:47.138047 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:47.138051 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:47.138078 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:47.140009 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.141657 (+ 1648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.141662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.141664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.141759 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.143011 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.143015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.143018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.143263 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:47.143418 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:47.144722 (+ 1304us) server_negotiation.cc:299] Negotiation successful
0809 04:42:47.144841 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 04:42:47.145025 21432 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58e1d241e53f46cc884fa7241cc75f03 in term 1.
I20260809 04:42:47.145387 21488 leader_election.cc:288] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [CANDIDATE]: Term 1 election: Requested vote from peers b9ba2cdac65343c684bccb6c4bce2aa1 (127.5.222.254:35179), d63260a3f97a487fb235be57740c257f (127.5.222.252:42603)
I20260809 04:42:47.146287 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.126511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35468 (local address 127.5.222.254:35179)
0809 04:42:47.126689 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:47.126694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:47.127964 (+ 1270us) server_negotiation.cc:407] Connection header received
0809 04:42:47.129216 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:47.129220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:47.129265 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:47.129326 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:47.132215 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.133044 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.135098 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.135475 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.135548 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.138443 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:47.138447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:47.138449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:47.138475 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:47.140782 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.142094 (+ 1312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.142097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.142099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.142191 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.144010 (+ 1819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.144013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.144015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.144239 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:47.144369 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:47.146012 (+ 1643us) server_negotiation.cc:299] Negotiation successful
0809 04:42:47.146128 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260809 04:42:47.146315 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:47.126875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50836 (local address 127.5.222.252:42603)
0809 04:42:47.126981 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:47.126988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:47.137414 (+ 10426us) server_negotiation.cc:407] Connection header received
0809 04:42:47.137597 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:47.137620 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:47.137686 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:47.137739 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:47.138576 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.139418 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.140610 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:47.140984 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:47.141062 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:47.142739 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:47.142743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:47.142745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:47.142777 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:47.144479 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.144958 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.144961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.144963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.145043 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:47.145398 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:47.145401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:47.145404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:47.145623 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:47.145724 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:47.146085 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 04:42:47.146187 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 04:42:47.147291 21487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58e1d241e53f46cc884fa7241cc75f03. Latest consensus state: current_term: 1 leader_uuid: "58e1d241e53f46cc884fa7241cc75f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } } }
I20260809 04:42:47.147459 21487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:47.157310 21510 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } } }
I20260809 04:42:47.157477 21510 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:42:47.160230 21524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:42:47.161223 21524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:42:47.167039 21432 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [term 1 FOLLOWER]: Refusing update from remote peer 58e1d241e53f46cc884fa7241cc75f03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:42:47.167663 21487 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:42:47.171401 21510 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58e1d241e53f46cc884fa7241cc75f03. Latest consensus state: current_term: 1 leader_uuid: "58e1d241e53f46cc884fa7241cc75f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } } }
I20260809 04:42:47.171535 21510 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:42:47.172523 21267 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [term 1 FOLLOWER]: Refusing update from remote peer 58e1d241e53f46cc884fa7241cc75f03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:42:47.173334 21487 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:42:47.177738 21470 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58e1d241e53f46cc884fa7241cc75f03. Latest consensus state: current_term: 1 leader_uuid: "58e1d241e53f46cc884fa7241cc75f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } } }
I20260809 04:42:47.177858 21470 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:42:47.180459 21487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "58e1d241e53f46cc884fa7241cc75f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } } }
I20260809 04:42:47.180573 21487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:47.181869 21510 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "58e1d241e53f46cc884fa7241cc75f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } } }
I20260809 04:42:47.181986 21510 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:42:47.182586 21525 mvcc.cc:205] Tried to move back new op lower bound from 7316482323108741120 to 7316482323029065728. Current Snapshot: MvccSnapshot[applied={T|T < 7316482323108741120}]
I20260809 04:42:47.182775 21527 mvcc.cc:205] Tried to move back new op lower bound from 7316482323108741120 to 7316482323029065728. Current Snapshot: MvccSnapshot[applied={T|T < 7316482323108741120}]
I20260809 04:42:47.189043 21488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "58e1d241e53f46cc884fa7241cc75f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } } }
I20260809 04:42:47.189179 21488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03 [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:47.190285 21470 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "58e1d241e53f46cc884fa7241cc75f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ba2cdac65343c684bccb6c4bce2aa1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 35179 } } peers { permanent_uuid: "58e1d241e53f46cc884fa7241cc75f03" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 38883 } } peers { permanent_uuid: "d63260a3f97a487fb235be57740c257f" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42603 } } }
I20260809 04:42:47.190395 21470 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:42:47.190979 21524 catalog_manager.cc:1347] Generated new cluster ID: 9cb2e3ad77b0434cb34ec4fe2afb2035
I20260809 04:42:47.191045 21524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:42:47.191182 21529 mvcc.cc:205] Tried to move back new op lower bound from 7316482323108741120 to 7316482323029065728. Current Snapshot: MvccSnapshot[applied={T|T < 7316482323108741120}]
I20260809 04:42:47.217425 21524 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:42:47.218711 21524 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:42:47.241772 21524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58e1d241e53f46cc884fa7241cc75f03: Generated new TSK 0
I20260809 04:42:47.242722 21524 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:42:48.122471 21485 catalog_manager.cc:1259] Loaded cluster ID: 9cb2e3ad77b0434cb34ec4fe2afb2035
I20260809 04:42:48.122573 21485 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1: loading cluster ID for follower catalog manager: success
I20260809 04:42:48.124454 21485 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1: acquiring CA information for follower catalog manager: success
I20260809 04:42:48.125778 21485 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b9ba2cdac65343c684bccb6c4bce2aa1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:42:48.145938 21521 catalog_manager.cc:1259] Loaded cluster ID: 9cb2e3ad77b0434cb34ec4fe2afb2035
I20260809 04:42:48.146013 21521 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f: loading cluster ID for follower catalog manager: success
I20260809 04:42:48.147636 21521 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f: acquiring CA information for follower catalog manager: success
I20260809 04:42:48.148748 21521 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d63260a3f97a487fb235be57740c257f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:42:48.319573 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 21221
I20260809 04:42:48.328632 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 21302
I20260809 04:42:48.337224 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 21387
2026-08-09T04:42:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3151 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:42:48 dist-test-slave-vxj2 krb5kdc[21542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:42:48 dist-test-slave-vxj2 krb5kdc[21542](info): set up 2 sockets
Aug 09 04:42:48 dist-test-slave-vxj2 krb5kdc[21542](info): commencing operation
krb5kdc: starting...
W20260809 04:42:49.445065 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:42:49 dist-test-slave-vxj2 krb5kdc[21542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:42:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:42:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:42:49.571797 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:36087
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:36087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:42:49.683805 21555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:49.684142 21555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:49.684229 21555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:49.688467 21555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:42:49.688556 21555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:49.688593 21555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:49.688633 21555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:42:49.688678 21555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:42:49.694223 21555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:36087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:36087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:49.695375 21555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:49.696420 21555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:49.705044 21560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:49.705044 21561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:49.705133 21564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:49.705533 21555 server_base.cc:1034] running on GCE node
I20260809 04:42:49.706089 21555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:49.707270 21555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:49.708475 21555 hybrid_clock.cc:648] HybridClock initialized: now 1786250569708446 us; error 43 us; skew 500 ppm
Aug 09 04:42:49 dist-test-slave-vxj2 krb5kdc[21542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250569, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:49.711443 21555 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:42:49.712462 21555 webserver.cc:460] Webserver started at http://127.5.222.254:43135/ using document root <none> and password file <none>
I20260809 04:42:49.713033 21555 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:49.713129 21555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:49.713394 21555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:42:49.715490 21555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "070fa2ddc8db49e28915525a3b902c0e"
format_stamp: "Formatted at 2026-08-09 04:42:49 on dist-test-slave-vxj2"
server_key: "bd759c8a769167e59d19ac09f78a6c76"
server_key_iv: "b7ef7a271fb65e4a35905b1f6eff0f12"
server_key_version: "encryptionkey@0"
I20260809 04:42:49.716070 21555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "070fa2ddc8db49e28915525a3b902c0e"
format_stamp: "Formatted at 2026-08-09 04:42:49 on dist-test-slave-vxj2"
server_key: "bd759c8a769167e59d19ac09f78a6c76"
server_key_iv: "b7ef7a271fb65e4a35905b1f6eff0f12"
server_key_version: "encryptionkey@0"
I20260809 04:42:49.719844 21555 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 04:42:49.722283 21571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:49.723531 21555 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:42:49.723671 21555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "070fa2ddc8db49e28915525a3b902c0e"
format_stamp: "Formatted at 2026-08-09 04:42:49 on dist-test-slave-vxj2"
server_key: "bd759c8a769167e59d19ac09f78a6c76"
server_key_iv: "b7ef7a271fb65e4a35905b1f6eff0f12"
server_key_version: "encryptionkey@0"
I20260809 04:42:49.723794 21555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:49.742667 21555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:49.746173 21555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:49.746398 21555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:49.753677 21555 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:36087
I20260809 04:42:49.753691 21629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:36087 every 8 connection(s)
I20260809 04:42:49.754675 21555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:42:49.757488 21630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:49.758050 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 21555
I20260809 04:42:49.758157 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:42:49.758526 6011 external_mini_cluster.cc:1428] Setting key 975fb6a05cbb4dcfb7338623dda0465c
I20260809 04:42:49.764436 21630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e: Bootstrap starting.
I20260809 04:42:49.766978 21630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:49.767755 21630 log.cc:824] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:42:49 dist-test-slave-vxj2 krb5kdc[21542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:49.769871 21630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e: No bootstrap required, opened a new log
I20260809 04:42:49.774446 21633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:49.759930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.5.222.254:36087)
0809 04:42:49.760355 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:49.760374 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:49.760418 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:42:49.761316 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:49.761340 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:49.761764 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:49.762132 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:49.762818 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:49.764303 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:49.765298 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:49.766020 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:49.766168 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:49.768932 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:49.768937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:49.768949 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:49.768974 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:49.771073 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:49.771579 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:49.771585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:49.771591 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:49.771742 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:49.772064 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:49.772069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:49.772072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:49.772629 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:49.772822 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:49.773105 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 04:42:49.773436 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":110,"threads_started":1}
I20260809 04:42:49.774518 21630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "070fa2ddc8db49e28915525a3b902c0e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36087 } }
I20260809 04:42:49.774778 21630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:49.774838 21630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 070fa2ddc8db49e28915525a3b902c0e, State: Initialized, Role: FOLLOWER
I20260809 04:42:49.775281 21630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "070fa2ddc8db49e28915525a3b902c0e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36087 } }
I20260809 04:42:49.775441 21630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:42:49.775512 21630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:42:49.775612 21630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:49.776612 21630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "070fa2ddc8db49e28915525a3b902c0e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36087 } }
I20260809 04:42:49.776999 21630 leader_election.cc:302] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 070fa2ddc8db49e28915525a3b902c0e; no voters:
I20260809 04:42:49.777338 21630 leader_election.cc:288] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:42:49.777468 21635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:42:49.777678 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:42:49.777757 21635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [term 1 LEADER]: Becoming Leader. State: Replica: 070fa2ddc8db49e28915525a3b902c0e, State: Running, Role: LEADER
I20260809 04:42:49.778090 21635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "070fa2ddc8db49e28915525a3b902c0e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36087 } }
I20260809 04:42:49.778684 21630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:42:49.779219 21635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 070fa2ddc8db49e28915525a3b902c0e. Latest consensus state: current_term: 1 leader_uuid: "070fa2ddc8db49e28915525a3b902c0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "070fa2ddc8db49e28915525a3b902c0e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36087 } } }
I20260809 04:42:49.779346 21635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:49.780146 21636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "070fa2ddc8db49e28915525a3b902c0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "070fa2ddc8db49e28915525a3b902c0e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36087 } } }
I20260809 04:42:49.780277 21636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:49.781250 21641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:42:49.783864 21641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:42:49.789484 21641 catalog_manager.cc:1347] Generated new cluster ID: 0667ca8c2ec64f6cbacfafd018688c13
I20260809 04:42:49.789561 21641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:42:49.796332 21641 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:42:49.797298 21641 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:42:49.807736 21641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 070fa2ddc8db49e28915525a3b902c0e: Generated new TSK 0
I20260809 04:42:49.808454 21641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:42:49.834532 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 21555
2026-08-09T04:42:49Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1497 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:42:49 dist-test-slave-vxj2 krb5kdc[21659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:42:49 dist-test-slave-vxj2 krb5kdc[21659](info): set up 2 sockets
Aug 09 04:42:49 dist-test-slave-vxj2 krb5kdc[21659](info): commencing operation
krb5kdc: starting...
W20260809 04:42:50.953743 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:42:51 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:42:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:42:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:42:51.079464 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:38293
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:38293
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:42:51.193650 21673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:51.193921 21673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:51.193961 21673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:51.198000 21673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:42:51.198051 21673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:51.198071 21673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:51.198088 21673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:42:51.198103 21673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:42:51.203835 21673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:38293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:38293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:51.204802 21673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:51.205844 21673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:51.214074 21680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:51.214082 21679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:51.214283 21673 server_base.cc:1034] running on GCE node
W20260809 04:42:51.214149 21683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:51.214908 21673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:51.215957 21673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:51.217188 21673 hybrid_clock.cc:648] HybridClock initialized: now 1786250571217129 us; error 45 us; skew 500 ppm
Aug 09 04:42:51 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:51.225031 21673 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:42:51.225942 21673 webserver.cc:460] Webserver started at http://127.5.222.254:40485/ using document root <none> and password file <none>
I20260809 04:42:51.226543 21673 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:51.226640 21673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:51.226869 21673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:42:51.228880 21673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "3f00aa306161467cac4c773f9995371f"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "402182a0a87db3802d66aadfd52a3448"
server_key_iv: "2d732c6377194ba1da05dfa3a5b67540"
server_key_version: "encryptionkey@0"
I20260809 04:42:51.229470 21673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "3f00aa306161467cac4c773f9995371f"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "402182a0a87db3802d66aadfd52a3448"
server_key_iv: "2d732c6377194ba1da05dfa3a5b67540"
server_key_version: "encryptionkey@0"
I20260809 04:42:51.233045 21673 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:42:51.235468 21690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:51.236745 21673 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:42:51.236902 21673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "3f00aa306161467cac4c773f9995371f"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "402182a0a87db3802d66aadfd52a3448"
server_key_iv: "2d732c6377194ba1da05dfa3a5b67540"
server_key_version: "encryptionkey@0"
I20260809 04:42:51.237018 21673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:51.249307 21673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:51.252108 21673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:51.252302 21673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:51.259317 21673 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:38293
I20260809 04:42:51.259598 21744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:38293 every 8 connection(s)
I20260809 04:42:51.260247 21673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:42:51.262967 21746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:51.266989 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 21673
I20260809 04:42:51.267112 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:42:51.267423 6011 external_mini_cluster.cc:1428] Setting key 6a0ba88a825799aa074c80f5ff001e62
I20260809 04:42:51.268895 21746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Bootstrap starting.
I20260809 04:42:51.271401 21746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:51.272125 21746 log.cc:824] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:51.274088 21746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: No bootstrap required, opened a new log
Aug 09 04:42:51 dist-test-slave-vxj2 krb5kdc[21659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:51.278517 21746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } }
I20260809 04:42:51.278935 21746 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:51.279052 21746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f00aa306161467cac4c773f9995371f, State: Initialized, Role: FOLLOWER
I20260809 04:42:51.279585 21746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } }
I20260809 04:42:51.279771 21746 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:42:51.279842 21746 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:42:51.279989 21746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:51.281528 21746 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } }
I20260809 04:42:51.281929 21746 leader_election.cc:302] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f00aa306161467cac4c773f9995371f; no voters:
I20260809 04:42:51.282303 21746 leader_election.cc:288] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:42:51.282394 21751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:42:51.282617 21751 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 1 LEADER]: Becoming Leader. State: Replica: 3f00aa306161467cac4c773f9995371f, State: Running, Role: LEADER
I20260809 04:42:51.282979 21751 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } }
I20260809 04:42:51.283391 21749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:51.268779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37560 (local address 127.5.222.254:38293)
0809 04:42:51.269215 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:51.269232 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:51.269267 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:42:51.269875 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:51.269890 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:51.270191 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:51.270467 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:51.271361 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:51.272448 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:51.273394 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:51.273860 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:51.274036 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:51.276752 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:51.276757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:51.276770 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:51.276796 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:51.278951 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:51.279518 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:51.279524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:51.279530 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:51.279675 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:51.280145 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:51.280151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:51.280155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:51.280764 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:51.280981 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:51.281907 (+ 926us) server_negotiation.cc:299] Negotiation successful
0809 04:42:51.282280 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":149,"threads_started":1}
I20260809 04:42:51.283658 21746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:42:51.284268 21751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f00aa306161467cac4c773f9995371f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } } }
I20260809 04:42:51.284389 21751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:51.284596 21751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f00aa306161467cac4c773f9995371f. Latest consensus state: current_term: 1 leader_uuid: "3f00aa306161467cac4c773f9995371f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } } }
I20260809 04:42:51.284678 21751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: This master's current role is: LEADER
W20260809 04:42:51.290282 21766 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:42:51.290458 21766 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:42:51.290612 21755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:42:51.291790 21755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:42:51.299685 21755 catalog_manager.cc:1347] Generated new cluster ID: 267cc2e146b94e8e8dcf367a521314a8
I20260809 04:42:51.299750 21755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:42:51.312254 21755 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:42:51.313484 21755 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:42:51.321491 21755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Generated new TSK 0
I20260809 04:42:51.322467 21755 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:42:51.361436 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:38293
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:42:51.480871 21772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:51.481132 21772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:51.481253 21772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:51.485267 21772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:51.485321 21772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:51.485415 21772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:42:51.490664 21772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:51.491663 21772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:51.492753 21772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:42:51.502357 21772 server_base.cc:1034] running on GCE node
W20260809 04:42:51.502280 21780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:51.502312 21778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:51.502270 21777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:51.502936 21772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:51.503629 21772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:51.504819 21772 hybrid_clock.cc:648] HybridClock initialized: now 1786250571504801 us; error 32 us; skew 500 ppm
Aug 09 04:42:51 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:51.507714 21772 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:42:51.508612 21772 webserver.cc:460] Webserver started at http://127.5.222.193:46189/ using document root <none> and password file <none>
I20260809 04:42:51.509230 21772 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:51.509297 21772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:51.509507 21772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:42:51.511497 21772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "339f6af8d9a344918e391a084ddd0ab3"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "b796548ca7f6b010e272dd932b9b97cc"
server_key_iv: "915250fd03606a47a825734bbfd235fd"
server_key_version: "encryptionkey@0"
I20260809 04:42:51.512075 21772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "339f6af8d9a344918e391a084ddd0ab3"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "b796548ca7f6b010e272dd932b9b97cc"
server_key_iv: "915250fd03606a47a825734bbfd235fd"
server_key_version: "encryptionkey@0"
I20260809 04:42:51.515673 21772 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 04:42:51.518054 21788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:51.519209 21772 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 04:42:51.519347 21772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "339f6af8d9a344918e391a084ddd0ab3"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "b796548ca7f6b010e272dd932b9b97cc"
server_key_iv: "915250fd03606a47a825734bbfd235fd"
server_key_version: "encryptionkey@0"
I20260809 04:42:51.519466 21772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:51.539788 21772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:51.543087 21772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:51.543295 21772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:51.543898 21772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:51.544734 21772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:42:51.544807 21772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:51.544878 21772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:42:51.544930 21772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:51.554677 21772 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:41333
I20260809 04:42:51.554711 21907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:41333 every 8 connection(s)
I20260809 04:42:51.555591 21772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:42:51.558138 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 21772
I20260809 04:42:51.558298 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:42:51.558672 6011 external_mini_cluster.cc:1428] Setting key 9dbc7ea68ddc9a3ac858f7b901b1bde6
Aug 09 04:42:51 dist-test-slave-vxj2 krb5kdc[21659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
I20260809 04:42:51.570480 21749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:51.557392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:39563 (local address 127.5.222.254:38293)
0809 04:42:51.557567 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:51.557571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:51.558525 (+ 954us) server_negotiation.cc:407] Connection header received
0809 04:42:51.560086 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:51.560090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:51.560141 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:51.560235 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:51.561871 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:51.562434 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:51.563616 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:51.563844 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:51.563959 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:51.566518 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:51.566524 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:51.566530 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:51.566570 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:51.568289 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:51.568804 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:51.568807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:51.568809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:51.568906 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:51.569307 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:51.569310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:51.569311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:51.569500 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:51.569613 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:51.570123 (+ 510us) server_negotiation.cc:299] Negotiation successful
0809 04:42:51.570290 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 04:42:51.571305 21910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:51.557745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:39563)
0809 04:42:51.558336 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:51.558399 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:51.559846 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:51.560446 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:51.560455 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:51.560973 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:51.561711 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:51.561733 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:51.562588 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:51.562592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:51.563475 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:51.563485 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:51.564012 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:51.564016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:51.564181 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:51.564811 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:51.564826 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:51.566318 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:51.568409 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:51.568416 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:51.568428 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:51.568677 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:51.568996 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:51.568999 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:51.569000 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:51.569127 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:51.569611 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:51.569617 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:51.569865 (+ 248us) client_negotiation.cc:769] Sending connection context
0809 04:42:51.570063 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 04:42:51.570395 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":124,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:42:51.572688 21908 heartbeater.cc:347] Connected to a master server at 127.5.222.254:38293
I20260809 04:42:51.573022 21908 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:51.573553 21908 heartbeater.cc:511] Master 127.5.222.254:38293 requested a full tablet report, sending...
I20260809 04:42:51.575141 21707 ts_manager.cc:194] Registered new tserver with Master: 339f6af8d9a344918e391a084ddd0ab3 (127.5.222.193:41333)
I20260809 04:42:51.576260 21707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:39563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:42:51.588603 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:38293
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:42:51.704118 21913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:51.704479 21913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:51.704530 21913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:51.708473 21913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:51.708537 21913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:51.708599 21913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:42:51.713892 21913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:51.715027 21913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:51.716020 21913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:51.725782 21918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:51.725818 21922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:51.725783 21919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:51.726306 21913 server_base.cc:1034] running on GCE node
I20260809 04:42:51.726708 21913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:51.727452 21913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:51.728643 21913 hybrid_clock.cc:648] HybridClock initialized: now 1786250571728629 us; error 25 us; skew 500 ppm
Aug 09 04:42:51 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:51.731671 21913 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:42:51.732647 21913 webserver.cc:460] Webserver started at http://127.5.222.194:44739/ using document root <none> and password file <none>
I20260809 04:42:51.733307 21913 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:51.733384 21913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:51.733618 21913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:42:51.735920 21913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "c2bc1810f85947818231f5d89ee41ed4"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "6f308f4a04159b6ed4b76a847bed7cec"
server_key_iv: "e0eabbc9f540b269c648f88ea026ce6e"
server_key_version: "encryptionkey@0"
I20260809 04:42:51.736505 21913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "c2bc1810f85947818231f5d89ee41ed4"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "6f308f4a04159b6ed4b76a847bed7cec"
server_key_iv: "e0eabbc9f540b269c648f88ea026ce6e"
server_key_version: "encryptionkey@0"
I20260809 04:42:51.740389 21913 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:42:51.742848 21929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:51.744051 21913 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:42:51.744171 21913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "c2bc1810f85947818231f5d89ee41ed4"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "6f308f4a04159b6ed4b76a847bed7cec"
server_key_iv: "e0eabbc9f540b269c648f88ea026ce6e"
server_key_version: "encryptionkey@0"
I20260809 04:42:51.744278 21913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:51.759989 21913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:51.762790 21913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:51.762988 21913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:51.763578 21913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:51.764394 21913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:42:51.764444 21913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:51.764510 21913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:42:51.764549 21913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:51.773559 21913 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:43109
I20260809 04:42:51.773586 22053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:43109 every 8 connection(s)
I20260809 04:42:51.774549 21913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:42:51.775544 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 21913
I20260809 04:42:51.775647 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:42:51.776022 6011 external_mini_cluster.cc:1428] Setting key 451aa5602e3fb144fe9d40ae51c756c6
Aug 09 04:42:51 dist-test-slave-vxj2 krb5kdc[21659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
I20260809 04:42:51.790838 21749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:51.776479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:46021 (local address 127.5.222.254:38293)
0809 04:42:51.776648 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:51.776652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:51.777594 (+ 942us) server_negotiation.cc:407] Connection header received
0809 04:42:51.779482 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:51.779485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:51.779530 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:51.779651 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:51.780837 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:51.781505 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:51.783402 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:51.783660 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:51.783749 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:51.786978 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:51.786983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:51.786986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:51.787017 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:51.788494 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:51.789101 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:51.789104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:51.789106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:51.789234 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:51.789590 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:51.789594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:51.789596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:51.789860 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:51.789980 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:51.790610 (+ 630us) server_negotiation.cc:299] Negotiation successful
0809 04:42:51.790711 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:42:51.792021 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:51.776905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:46021)
0809 04:42:51.777432 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:51.777492 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:51.779248 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:51.779864 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:51.779871 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:51.780227 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:51.780689 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:51.780702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:51.781636 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:51.781643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:51.783256 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:51.783271 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:51.783835 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:51.783841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:51.784092 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:51.784835 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:51.784860 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:51.786788 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:51.788605 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:51.788609 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:51.788621 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:51.788987 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:51.789323 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:51.789326 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:51.789327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:51.789461 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:51.789991 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:51.789997 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:51.790339 (+ 342us) client_negotiation.cc:769] Sending connection context
0809 04:42:51.790571 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 04:42:51.790992 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":116,"threads_started":1}
I20260809 04:42:51.793224 22054 heartbeater.cc:347] Connected to a master server at 127.5.222.254:38293
I20260809 04:42:51.793475 22054 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:51.794096 22054 heartbeater.cc:511] Master 127.5.222.254:38293 requested a full tablet report, sending...
I20260809 04:42:51.795312 21707 ts_manager.cc:194] Registered new tserver with Master: c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109)
I20260809 04:42:51.795908 21707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:46021
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:42:51.807653 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:38293
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:42:51.920235 22059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:51.920562 22059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:51.920634 22059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:51.924986 22059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:51.925071 22059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:51.925180 22059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:42:51.930521 22059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:51.931640 22059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:51.932677 22059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:51.942934 22066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:51.942904 22065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:51.942989 22068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:51.943058 22059 server_base.cc:1034] running on GCE node
I20260809 04:42:51.943594 22059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:51.944267 22059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:51.945454 22059 hybrid_clock.cc:648] HybridClock initialized: now 1786250571945435 us; error 29 us; skew 500 ppm
Aug 09 04:42:51 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:51.948482 22059 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:42:51.949369 22059 webserver.cc:460] Webserver started at http://127.5.222.195:45607/ using document root <none> and password file <none>
I20260809 04:42:51.949880 22059 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:51.949937 22059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:51.950098 22059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:42:51.952031 22059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "e6324db4e82b426fb0a26b7178377638"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "1db1be8d871f2fa4c86d16b7992f0859"
server_key_iv: "2b5db305f5b44952e38d0ea568a77ad8"
server_key_version: "encryptionkey@0"
I20260809 04:42:51.952521 22059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "e6324db4e82b426fb0a26b7178377638"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "1db1be8d871f2fa4c86d16b7992f0859"
server_key_iv: "2b5db305f5b44952e38d0ea568a77ad8"
server_key_version: "encryptionkey@0"
I20260809 04:42:51.956200 22059 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 04:42:51.958636 22075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:51.959923 22059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:42:51.960026 22059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "e6324db4e82b426fb0a26b7178377638"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "1db1be8d871f2fa4c86d16b7992f0859"
server_key_iv: "2b5db305f5b44952e38d0ea568a77ad8"
server_key_version: "encryptionkey@0"
I20260809 04:42:51.960112 22059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:51.992208 22059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:51.995298 22059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:51.995452 22059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:51.995993 22059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:51.996766 22059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:42:51.996814 22059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:51.996855 22059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:42:51.996869 22059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:52.005949 22059 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:33693
I20260809 04:42:52.006036 22196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:33693 every 8 connection(s)
I20260809 04:42:52.006912 22059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:42:52.014569 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 22059
I20260809 04:42:52.014667 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:42:52.014954 6011 external_mini_cluster.cc:1428] Setting key 379b94a7ad35058ee2473c9db3052273
Aug 09 04:42:52 dist-test-slave-vxj2 krb5kdc[21659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:52.021193 21749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.008839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:60777 (local address 127.5.222.254:38293)
0809 04:42:52.009007 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:52.009011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:52.009957 (+ 946us) server_negotiation.cc:407] Connection header received
0809 04:42:52.011280 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:52.011283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:52.011342 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:52.011457 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:52.012651 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.013252 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.014425 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.014685 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.014794 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.017845 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:52.017848 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:52.017851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:52.017877 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:52.019142 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:52.019656 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:52.019661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:52.019663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:52.019748 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:52.020083 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:52.020087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:52.020089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:52.020281 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:52.020391 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:52.020882 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 04:42:52.021004 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 04:42:52.021944 22199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.009264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:60777)
0809 04:42:52.009795 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:52.009837 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:52.011061 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:52.011637 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:52.011643 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:52.012001 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:52.012493 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.012507 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.013377 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.013386 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.014259 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.014268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.014835 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.014839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.015006 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.015821 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:52.015844 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:52.017671 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:52.019283 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:52.019288 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:52.019303 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:52.019552 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:52.019841 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:52.019845 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:52.019846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:52.019986 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:52.020388 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:52.020395 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:52.020630 (+ 235us) client_negotiation.cc:769] Sending connection context
0809 04:42:52.020818 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 04:42:52.021126 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":168,"threads_started":1}
I20260809 04:42:52.022967 22197 heartbeater.cc:347] Connected to a master server at 127.5.222.254:38293
I20260809 04:42:52.023217 22197 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:52.023643 22197 heartbeater.cc:511] Master 127.5.222.254:38293 requested a full tablet report, sending...
I20260809 04:42:52.024569 21707 ts_manager.cc:194] Registered new tserver with Master: e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693)
I20260809 04:42:52.025065 21707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:60777
I20260809 04:42:52.028743 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:42:52.038002 21749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.030713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37562 (local address 127.5.222.254:38293)
0809 04:42:52.030850 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:52.030854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:52.031036 (+ 182us) server_negotiation.cc:407] Connection header received
0809 04:42:52.031125 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:52.031127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:52.031163 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:52.031228 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:52.032008 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.032524 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.033497 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.033701 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.033805 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.035158 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:52.035161 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:52.035163 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:52.035183 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:52.036407 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:52.036798 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:52.036801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:52.036802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:52.036887 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:52.037122 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:52.037124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:52.037125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:52.037389 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:52.037495 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:52.037698 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 04:42:52.037793 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:42:52.040699 21707 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:42:52.043845 21707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:42:52.058530 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.052309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43109 (local address 127.0.0.1:45392)
0809 04:42:52.053080 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:52.053097 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:52.053213 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:52.053895 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:52.053901 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:52.053931 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:52.054029 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.054035 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.055460 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.055463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.056928 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.056939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.057975 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.057978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.058089 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.058124 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 04:42:52.058233 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 04:42:52.058347 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":123,"thread_start_us":114,"threads_started":1}
I20260809 04:42:52.058658 22210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.052280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.5.222.195:33693)
0809 04:42:52.053200 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:52.053212 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:52.053253 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:42:52.053336 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:52.053344 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:52.053551 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:52.054086 (+ 535us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:52.054268 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.055157 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.057245 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.058110 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.058224 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.058355 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:52.058416 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 04:42:52.058503 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":161,"threads_started":1}
I20260809 04:42:52.058548 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.052117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:33693 (local address 127.0.0.1:47484)
0809 04:42:52.052580 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:52.052629 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:52.052793 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:52.053727 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:52.053733 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:52.053757 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:52.053928 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.053936 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.055281 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.055286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.056928 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.056939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.058224 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.058230 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.058309 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.058333 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:42:52.058390 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 04:42:52.058448 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":136,"thread_start_us":65,"threads_started":1}
I20260809 04:42:52.059017 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.052827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:41333 (local address 127.0.0.1:51626)
0809 04:42:52.054160 (+ 1333us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:52.054173 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:52.054280 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:52.054712 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:52.054722 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:52.054740 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:52.054826 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.054832 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.055923 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.055929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.057227 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.057238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.058601 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.058607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.058715 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.058747 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:42:52.058813 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 04:42:52.058871 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1271,"mutex_wait_us":20,"thread_start_us":88,"threads_started":1}
I20260809 04:42:52.059329 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.052822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45392 (local address 127.5.222.194:43109)
0809 04:42:52.053460 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:52.053471 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:52.053491 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:42:52.053559 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:52.053566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:52.053747 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:52.053903 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:52.054153 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.055327 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.057060 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.057759 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.057980 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.058131 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:52.059084 (+ 953us) server_negotiation.cc:299] Negotiation successful
0809 04:42:52.059192 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":160,"threads_started":1}
I20260809 04:42:52.060149 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.053061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51626 (local address 127.5.222.193:41333)
0809 04:42:52.053728 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:52.053745 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:52.054283 (+ 538us) server_negotiation.cc:407] Connection header received
0809 04:42:52.054389 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:52.054394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:52.054567 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:52.054710 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:52.054926 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.055795 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.057354 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.058119 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.058232 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.058356 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:52.059152 (+ 796us) server_negotiation.cc:299] Negotiation successful
0809 04:42:52.059987 (+ 835us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":142,"threads_started":1}
I20260809 04:42:52.062103 21981 tablet_service.cc:1467] Processing CreateTablet for tablet 9754972977234caf87df45f68b11b0dc (DEFAULT_TABLE table=test-table [id=beafb62b89ba4aa0b1c500fa99974d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:42:52.062103 22125 tablet_service.cc:1467] Processing CreateTablet for tablet 9754972977234caf87df45f68b11b0dc (DEFAULT_TABLE table=test-table [id=beafb62b89ba4aa0b1c500fa99974d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:42:52.062685 21838 tablet_service.cc:1467] Processing CreateTablet for tablet 9754972977234caf87df45f68b11b0dc (DEFAULT_TABLE table=test-table [id=beafb62b89ba4aa0b1c500fa99974d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:42:52.063235 22125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9754972977234caf87df45f68b11b0dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:52.063235 21981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9754972977234caf87df45f68b11b0dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:52.063680 21838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9754972977234caf87df45f68b11b0dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:52.069772 22214 tablet_bootstrap.cc:492] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Bootstrap starting.
I20260809 04:42:52.070683 22215 tablet_bootstrap.cc:492] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Bootstrap starting.
I20260809 04:42:52.072067 22214 tablet_bootstrap.cc:654] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:52.072732 22215 tablet_bootstrap.cc:654] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:52.072819 22214 log.cc:824] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:52.073489 22216 tablet_bootstrap.cc:492] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638: Bootstrap starting.
I20260809 04:42:52.073550 22215 log.cc:824] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:52.074791 22214 tablet_bootstrap.cc:492] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: No bootstrap required, opened a new log
I20260809 04:42:52.075017 22214 ts_tablet_manager.cc:1397] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260809 04:42:52.075469 22215 tablet_bootstrap.cc:492] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: No bootstrap required, opened a new log
I20260809 04:42:52.075663 22215 ts_tablet_manager.cc:1397] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:42:52.076036 22216 tablet_bootstrap.cc:654] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:52.076798 22216 log.cc:824] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:52.078547 22214 raft_consensus.cc:348] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.078789 22216 tablet_bootstrap.cc:492] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638: No bootstrap required, opened a new log
I20260809 04:42:52.078796 22214 raft_consensus.cc:374] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:52.078847 22214 raft_consensus.cc:729] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339f6af8d9a344918e391a084ddd0ab3, State: Initialized, Role: FOLLOWER
I20260809 04:42:52.079022 22216 ts_tablet_manager.cc:1397] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:42:52.079355 22214 consensus_queue.cc:260] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.079533 22215 raft_consensus.cc:348] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.079842 22215 raft_consensus.cc:374] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:52.079936 22215 raft_consensus.cc:729] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2bc1810f85947818231f5d89ee41ed4, State: Initialized, Role: FOLLOWER
I20260809 04:42:52.080019 21908 heartbeater.cc:503] Master 127.5.222.254:38293 was elected leader, sending a full tablet report...
I20260809 04:42:52.080365 22214 ts_tablet_manager.cc:1428] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260809 04:42:52.080438 22215 consensus_queue.cc:260] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.081418 22215 ts_tablet_manager.cc:1428] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:42:52.081650 22054 heartbeater.cc:503] Master 127.5.222.254:38293 was elected leader, sending a full tablet report...
I20260809 04:42:52.082706 22216 raft_consensus.cc:348] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.083015 22216 raft_consensus.cc:374] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:52.083092 22216 raft_consensus.cc:729] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6324db4e82b426fb0a26b7178377638, State: Initialized, Role: FOLLOWER
I20260809 04:42:52.083529 22216 consensus_queue.cc:260] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.084403 22216 ts_tablet_manager.cc:1428] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 04:42:52.084728 22197 heartbeater.cc:503] Master 127.5.222.254:38293 was elected leader, sending a full tablet report...
I20260809 04:42:52.205083 22222 raft_consensus.cc:482] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:42:52.205325 22222 raft_consensus.cc:504] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.206377 22222 leader_election.cc:288] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109), 339f6af8d9a344918e391a084ddd0ab3 (127.5.222.193:41333)
I20260809 04:42:52.210158 22199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.206640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43109 (local address 127.5.222.195:58207)
0809 04:42:52.206774 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:52.206792 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:52.206886 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:52.207223 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:52.207227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:52.207246 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:52.207373 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.207379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.208153 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.208157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.208946 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.208955 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.209677 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.209714 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.209844 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.209863 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 04:42:52.209956 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 04:42:52.210024 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 04:42:52.210160 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.206815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:58207 (local address 127.5.222.194:43109)
0809 04:42:52.206959 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:52.206965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:52.206983 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:42:52.207047 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:52.207050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:52.207102 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:52.207207 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:52.207496 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.208024 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.209089 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.209551 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.209686 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.209762 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:52.209973 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 04:42:52.210056 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:42:52.210537 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.207134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:51679 (local address 127.5.222.193:41333)
0809 04:42:52.207268 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:52.207273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:52.207475 (+ 202us) server_negotiation.cc:407] Connection header received
0809 04:42:52.207594 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:52.207599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:52.207647 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:52.207735 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:52.208039 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.208537 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.209585 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.209974 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.210101 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.210148 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:52.210354 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 04:42:52.210422 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:42:52.210886 22223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.206943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:41333 (local address 127.5.222.195:51679)
0809 04:42:52.207380 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:52.207391 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:52.207505 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:52.207743 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:52.207746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:52.207760 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:52.207897 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.207903 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.208653 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.208658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.209457 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.209466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.210093 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.210097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.210194 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.210219 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:42:52.210694 (+ 475us) client_negotiation.cc:240] Negotiation successful
0809 04:42:52.210769 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":160,"threads_started":1}
I20260809 04:42:52.210875 22002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9754972977234caf87df45f68b11b0dc" candidate_uuid: "e6324db4e82b426fb0a26b7178377638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2bc1810f85947818231f5d89ee41ed4" is_pre_election: true
I20260809 04:42:52.211205 22002 raft_consensus.cc:2454] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6324db4e82b426fb0a26b7178377638 in term 0.
I20260809 04:42:52.211508 21859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9754972977234caf87df45f68b11b0dc" candidate_uuid: "e6324db4e82b426fb0a26b7178377638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "339f6af8d9a344918e391a084ddd0ab3" is_pre_election: true
I20260809 04:42:52.211687 22078 leader_election.cc:302] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2bc1810f85947818231f5d89ee41ed4, e6324db4e82b426fb0a26b7178377638; no voters:
I20260809 04:42:52.211776 21859 raft_consensus.cc:2454] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6324db4e82b426fb0a26b7178377638 in term 0.
I20260809 04:42:52.211971 22222 raft_consensus.cc:2781] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:42:52.212061 22222 raft_consensus.cc:482] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:42:52.212153 22222 raft_consensus.cc:3037] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:52.213244 22222 raft_consensus.cc:504] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.213579 22222 leader_election.cc:288] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [CANDIDATE]: Term 1 election: Requested vote from peers c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109), 339f6af8d9a344918e391a084ddd0ab3 (127.5.222.193:41333)
I20260809 04:42:52.213966 21859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9754972977234caf87df45f68b11b0dc" candidate_uuid: "e6324db4e82b426fb0a26b7178377638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "339f6af8d9a344918e391a084ddd0ab3"
I20260809 04:42:52.213963 22002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9754972977234caf87df45f68b11b0dc" candidate_uuid: "e6324db4e82b426fb0a26b7178377638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2bc1810f85947818231f5d89ee41ed4"
I20260809 04:42:52.214089 22002 raft_consensus.cc:3037] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:52.214089 21859 raft_consensus.cc:3037] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:52.215412 22002 raft_consensus.cc:2454] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6324db4e82b426fb0a26b7178377638 in term 1.
I20260809 04:42:52.215412 21859 raft_consensus.cc:2454] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6324db4e82b426fb0a26b7178377638 in term 1.
I20260809 04:42:52.215704 22078 leader_election.cc:302] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339f6af8d9a344918e391a084ddd0ab3, e6324db4e82b426fb0a26b7178377638; no voters:
I20260809 04:42:52.215914 22222 raft_consensus.cc:2781] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:42:52.216171 22222 raft_consensus.cc:686] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 1 LEADER]: Becoming Leader. State: Replica: e6324db4e82b426fb0a26b7178377638, State: Running, Role: LEADER
I20260809 04:42:52.216499 22222 consensus_queue.cc:237] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.219724 21707 catalog_manager.cc:5495] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6324db4e82b426fb0a26b7178377638 (127.5.222.195). New cstate: current_term: 1 leader_uuid: "e6324db4e82b426fb0a26b7178377638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } health_report { overall_health: UNKNOWN } } }
I20260809 04:42:52.248086 22210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.244330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47494 (local address 127.5.222.195:33693)
0809 04:42:52.244466 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:52.244470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:52.244483 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:42:52.244628 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:52.244631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:52.244684 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:52.244785 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:42:52.245062 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.245625 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.246451 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.246654 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.246729 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.247120 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:42:52.247561 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:42:52.247678 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:52.247845 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 04:42:52.247896 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:42:52.253979 21859 raft_consensus.cc:1261] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Refusing update from remote peer e6324db4e82b426fb0a26b7178377638: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:42:52.253979 22002 raft_consensus.cc:1261] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 1 FOLLOWER]: Refusing update from remote peer e6324db4e82b426fb0a26b7178377638: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:42:52.254720 22224 consensus_queue.cc:1035] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:42:52.254882 22222 consensus_queue.cc:1035] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 04:42:52.257820 22198 tablet.cc:2369] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:42:52.265616 22229 mvcc.cc:205] Tried to move back new op lower bound from 7316482343944843264 to 7316482343801274368. Current Snapshot: MvccSnapshot[applied={T|T < 7316482343944843264}]
I20260809 04:42:52.272203 21707 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:42:52.272370 21707 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:42:52.274582 21707 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Sending DeleteTablet for 3 replicas of tablet 9754972977234caf87df45f68b11b0dc
I20260809 04:42:52.275482 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 21772
I20260809 04:42:52.275451 22125 tablet_service.cc:1514] Processing DeleteTablet for tablet 9754972977234caf87df45f68b11b0dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:42:52 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:47484
I20260809 04:42:52.275430 21838 tablet_service.cc:1514] Processing DeleteTablet for tablet 9754972977234caf87df45f68b11b0dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:42:52 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:51626
I20260809 04:42:52.275924 21981 tablet_service.cc:1514] Processing DeleteTablet for tablet 9754972977234caf87df45f68b11b0dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:42:52 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:45392
I20260809 04:42:52.280571 22237 tablet_replica.cc:330] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638: stopping tablet replica
I20260809 04:42:52.280907 22237 raft_consensus.cc:2229] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:42:52.281328 22239 tablet_replica.cc:330] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: stopping tablet replica
I20260809 04:42:52.281600 22239 raft_consensus.cc:2229] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:42:52.281945 22239 raft_consensus.cc:2258] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:42:52.282263 22237 raft_consensus.cc:2258] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 04:42:52.283715 21693 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:41333 (error 108)
I20260809 04:42:52.284128 22237 ts_tablet_manager.cc:1905] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:42:52.284612 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 21913
I20260809 04:42:52.284718 22239 ts_tablet_manager.cc:1905] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:42:52.291806 22237 ts_tablet_manager.cc:1918] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:42:52.291956 22237 log.cc:1196] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/9754972977234caf87df45f68b11b0dc
I20260809 04:42:52.292353 22237 ts_tablet_manager.cc:1939] T 9754972977234caf87df45f68b11b0dc P e6324db4e82b426fb0a26b7178377638: Deleting consensus metadata
I20260809 04:42:52.293725 21692 catalog_manager.cc:4841] TS e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693): tablet 9754972977234caf87df45f68b11b0dc (table test-table [id=beafb62b89ba4aa0b1c500fa99974d2d]) successfully deleted
I20260809 04:42:52.297000 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.296636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:41333 (local address 127.0.0.1:51636)
0809 04:42:52.296808 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:52.296899 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:41333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 04:42:52.297262 21693 catalog_manager.cc:4584] TS 339f6af8d9a344918e391a084ddd0ab3 (127.5.222.193:41333): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9754972977234caf87df45f68b11b0dc: Network error: Client connection negotiation failed: client connection to 127.5.222.193:41333: connect: Connection refused (error 111)
I20260809 04:42:52.299844 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 22059
I20260809 04:42:52.302647 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.300007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43109 (local address 127.0.0.1:45394)
0809 04:42:52.302482 (+ 2475us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:52.302549 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.194:43109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2361}
I20260809 04:42:52.306700 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 21673
I20260809 04:42:52.315224 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:38293
--webserver_interface=127.5.222.254
--webserver_port=40485
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:38293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:42:52.426992 22241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:52.427414 22241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:52.427476 22241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:52.434870 22241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:42:52.434959 22241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:52.434995 22241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:52.435032 22241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:42:52.435066 22241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:42:52.440704 22241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:38293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:38293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:52.441871 22241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:52.442888 22241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:52.459486 22248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:52.459678 22241 server_base.cc:1034] running on GCE node
W20260809 04:42:52.459509 22247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:52.459893 22250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:52.460585 22241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:52.461856 22241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:52.463054 22241 hybrid_clock.cc:648] HybridClock initialized: now 1786250572463021 us; error 55 us; skew 500 ppm
Aug 09 04:42:52 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:52.466203 22241 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:42:52.467163 22241 webserver.cc:460] Webserver started at http://127.5.222.254:40485/ using document root <none> and password file <none>
I20260809 04:42:52.467753 22241 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:52.467861 22241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:52.471983 22241 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:42:52.474372 22257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:52.475633 22241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:42:52.475781 22241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "3f00aa306161467cac4c773f9995371f"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "402182a0a87db3802d66aadfd52a3448"
server_key_iv: "2d732c6377194ba1da05dfa3a5b67540"
server_key_version: "encryptionkey@0"
I20260809 04:42:52.476399 22241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:52.504873 22241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:52.508159 22241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:52.508414 22241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:52.516448 22241 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:38293
I20260809 04:42:52.516752 22314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:38293 every 8 connection(s)
I20260809 04:42:52.517774 22241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:42:52.522059 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 22241
I20260809 04:42:52.522413 22315 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:42:52.522584 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:41333
--local_ip_for_outbound_sockets=127.5.222.193
--tserver_master_addrs=127.5.222.254:38293
--webserver_port=46189
--webserver_interface=127.5.222.193
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:42:52.524164 22315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Bootstrap starting.
I20260809 04:42:52.541955 22315 log.cc:824] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:52.550884 22315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:42:52.551282 22315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Bootstrap complete.
I20260809 04:42:52.555488 22315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } }
I20260809 04:42:52.555927 22315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f00aa306161467cac4c773f9995371f, State: Initialized, Role: FOLLOWER
I20260809 04:42:52.556468 22315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } }
I20260809 04:42:52.556617 22315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:42:52.556718 22315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:42:52.556847 22315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:42:52.558868 22315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } }
I20260809 04:42:52.559203 22315 leader_election.cc:302] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f00aa306161467cac4c773f9995371f; no voters:
I20260809 04:42:52.559545 22315 leader_election.cc:288] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:42:52.559640 22319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:42:52.559844 22319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 2 LEADER]: Becoming Leader. State: Replica: 3f00aa306161467cac4c773f9995371f, State: Running, Role: LEADER
I20260809 04:42:52.560135 22319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } }
I20260809 04:42:52.560820 22315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:42:52.561489 22320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3f00aa306161467cac4c773f9995371f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } } }
I20260809 04:42:52.561627 22320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:52.562240 22321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f00aa306161467cac4c773f9995371f. Latest consensus state: current_term: 2 leader_uuid: "3f00aa306161467cac4c773f9995371f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } } }
I20260809 04:42:52.562352 22321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:52.562304 22326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:42:52.567466 22326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:42:52.568229 22326 catalog_manager.cc:1259] Loaded cluster ID: 267cc2e146b94e8e8dcf367a521314a8
I20260809 04:42:52.568306 22326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:42:52.569844 22326 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 04:42:52.574504 22326 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 04:42:52.580232 22326 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:42:52.581278 22326 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Loaded TSK: 0
I20260809 04:42:52.582538 22326 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:42:52.645015 22317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:52.645287 22317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:52.645367 22317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:52.649340 22317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:52.649392 22317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:52.649483 22317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:42:52.654711 22317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:41333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=46189
--tserver_master_addrs=127.5.222.254:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:52.655799 22317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:52.656787 22317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:52.666369 22342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:52.666365 22346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:52.666419 22343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:52.666910 22317 server_base.cc:1034] running on GCE node
I20260809 04:42:52.667373 22317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:52.668130 22317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:52.669337 22317 hybrid_clock.cc:648] HybridClock initialized: now 1786250572669305 us; error 40 us; skew 500 ppm
Aug 09 04:42:52 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:52.672776 22317 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:42:52.673954 22317 webserver.cc:460] Webserver started at http://127.5.222.193:46189/ using document root <none> and password file <none>
I20260809 04:42:52.674664 22317 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:52.674782 22317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:52.679993 22317 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:42:52.682607 22355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:52.684103 22317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:42:52.684226 22317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "339f6af8d9a344918e391a084ddd0ab3"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "b796548ca7f6b010e272dd932b9b97cc"
server_key_iv: "915250fd03606a47a825734bbfd235fd"
server_key_version: "encryptionkey@0"
I20260809 04:42:52.684933 22317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:52.714555 22317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:52.718072 22317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:52.718283 22317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:52.718945 22317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:52.720219 22362 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 04:42:52.724296 22317 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 04:42:52.724356 22317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260809 04:42:52.724406 22317 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 04:42:52.725939 22317 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 04:42:52.725983 22317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260809 04:42:52.726297 22362 tablet_bootstrap.cc:492] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Bootstrap starting.
I20260809 04:42:52.736763 22317 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:41333
I20260809 04:42:52.736836 22474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:41333 every 8 connection(s)
I20260809 04:42:52.737772 22317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:42:52.739996 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 22317
I20260809 04:42:52.740504 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:43109
--local_ip_for_outbound_sockets=127.5.222.194
--tserver_master_addrs=127.5.222.254:38293
--webserver_port=44739
--webserver_interface=127.5.222.194
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:42:52.745946 22362 log.cc:824] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:42:52 dist-test-slave-vxj2 krb5kdc[21659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:52.755527 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.739977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:39263)
0809 04:42:52.741229 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:52.741313 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:52.742426 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:52.743555 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:52.743563 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:52.743988 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:52.744587 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.744603 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.746037 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.746058 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.747204 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.747214 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.747966 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.747970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.748149 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.748875 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:52.748890 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:52.750408 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:52.752544 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:52.752549 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:52.752561 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:52.752823 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:52.753196 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:52.753201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:52.753203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:52.753346 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:52.753906 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:52.753912 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:52.754146 (+ 234us) client_negotiation.cc:769] Sending connection context
0809 04:42:52.754322 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 04:42:52.754645 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"thread_start_us":176,"threads_started":1}
I20260809 04:42:52.755517 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.740018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:39263 (local address 127.5.222.254:38293)
0809 04:42:52.740453 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:52.740475 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:52.741470 (+ 995us) server_negotiation.cc:407] Connection header received
0809 04:42:52.742868 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:52.742880 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:52.743170 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:52.743503 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:52.744778 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.745852 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.747338 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.747811 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.747998 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.750609 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:52.750614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:52.750625 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:52.750652 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:52.752397 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:52.752944 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:52.752950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:52.752962 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:52.753077 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:52.753445 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:52.753450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:52.753451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:52.753790 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:52.753933 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:52.754451 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 04:42:52.754722 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":145,"threads_started":1}
I20260809 04:42:52.756088 22362 tablet_bootstrap.cc:492] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:42:52.756507 22362 tablet_bootstrap.cc:492] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Bootstrap complete.
I20260809 04:42:52.757977 22362 ts_tablet_manager.cc:1397] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Time spent bootstrapping tablet: real 0.032s user 0.015s sys 0.015s
I20260809 04:42:52.758251 22475 heartbeater.cc:347] Connected to a master server at 127.5.222.254:38293
I20260809 04:42:52.758482 22475 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:52.758941 22475 heartbeater.cc:511] Master 127.5.222.254:38293 requested a full tablet report, sending...
I20260809 04:42:52.760779 22362 raft_consensus.cc:348] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.761310 22362 raft_consensus.cc:729] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339f6af8d9a344918e391a084ddd0ab3, State: Initialized, Role: FOLLOWER
I20260809 04:42:52.761319 22277 ts_manager.cc:194] Registered new tserver with Master: 339f6af8d9a344918e391a084ddd0ab3 (127.5.222.193:41333)
I20260809 04:42:52.761770 22362 consensus_queue.cc:260] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.762789 22362 ts_tablet_manager.cc:1428] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:42:52.764820 22277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:39263
I20260809 04:42:52.767346 22475 heartbeater.cc:503] Master 127.5.222.254:38293 was elected leader, sending a full tablet report...
Aug 09 04:42:52 dist-test-slave-vxj2 krb5kdc[21659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.193@KRBTEST.COM
I20260809 04:42:52.775215 22483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.764264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51642 (local address 127.5.222.193:41333)
0809 04:42:52.764608 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:52.764618 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:52.765053 (+ 435us) server_negotiation.cc:407] Connection header received
0809 04:42:52.765281 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:52.765287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:52.765475 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:52.765624 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:52.766328 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.767336 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.768551 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.769006 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.769089 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.771668 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:52.771677 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:52.771686 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:52.771707 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:52.773398 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:52.773915 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:52.773920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:52.773921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:52.773999 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:52.774293 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:52.774295 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:52.774296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:52.774504 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:52.774601 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:52.774969 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 04:42:52.775097 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":104,"threads_started":1}
I20260809 04:42:52.775255 22482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.764170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:41333 (local address 127.0.0.1:51642)
0809 04:42:52.764886 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:52.764950 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:52.765108 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:52.765640 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:52.765645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:52.766069 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:52.766194 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.766201 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.767453 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.767456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.768416 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.768429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.769122 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.769125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.769262 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.769777 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:52.769792 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:52.771515 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:52.773501 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:52.773510 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:52.773515 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:52.773801 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:52.774089 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:52.774092 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:52.774094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:52.774188 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:52.774614 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:52.774620 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:52.774746 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 04:42:52.774895 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 04:42:52.775091 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":112,"threads_started":1}
I20260809 04:42:52.776237 22405 tablet_service.cc:1514] Processing DeleteTablet for tablet 9754972977234caf87df45f68b11b0dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:42:52 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:51642
I20260809 04:42:52.777230 22484 tablet_replica.cc:330] stopping tablet replica
I20260809 04:42:52.777544 22484 raft_consensus.cc:2229] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:42:52.777810 22484 raft_consensus.cc:2258] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:42:52.778793 22484 ts_tablet_manager.cc:1905] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:42:52.782541 22484 ts_tablet_manager.cc:1918] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:42:52.782656 22484 log.cc:1196] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal/wals/9754972977234caf87df45f68b11b0dc
I20260809 04:42:52.782964 22484 ts_tablet_manager.cc:1939] T 9754972977234caf87df45f68b11b0dc P 339f6af8d9a344918e391a084ddd0ab3: Deleting consensus metadata
I20260809 04:42:52.784571 22260 catalog_manager.cc:4841] TS 339f6af8d9a344918e391a084ddd0ab3 (127.5.222.193:41333): tablet 9754972977234caf87df45f68b11b0dc (table test-table [id=beafb62b89ba4aa0b1c500fa99974d2d]) successfully deleted
W20260809 04:42:52.862576 22479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:52.862926 22479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:52.862998 22479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:52.867100 22479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:52.867162 22479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:52.867223 22479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:42:52.872360 22479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:43109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=44739
--tserver_master_addrs=127.5.222.254:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:52.873413 22479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:52.874420 22479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:42:52.884011 22479 server_base.cc:1034] running on GCE node
W20260809 04:42:52.883978 22490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:52.883944 22492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:52.883963 22494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:52.884605 22479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:52.885324 22479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:52.886507 22479 hybrid_clock.cc:648] HybridClock initialized: now 1786250572886494 us; error 26 us; skew 500 ppm
Aug 09 04:42:52 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:52.889326 22479 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:42:52.890267 22479 webserver.cc:460] Webserver started at http://127.5.222.194:44739/ using document root <none> and password file <none>
I20260809 04:42:52.890904 22479 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:52.891004 22479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:52.894673 22479 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 04:42:52.896806 22501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:52.898029 22479 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:42:52.898161 22479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "c2bc1810f85947818231f5d89ee41ed4"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "6f308f4a04159b6ed4b76a847bed7cec"
server_key_iv: "e0eabbc9f540b269c648f88ea026ce6e"
server_key_version: "encryptionkey@0"
I20260809 04:42:52.898694 22479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:52.911043 22479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:52.914247 22479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:52.914462 22479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:52.915052 22479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:52.916095 22510 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 04:42:52.920077 22479 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 04:42:52.920176 22479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260809 04:42:52.920255 22479 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 04:42:52.922015 22479 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 04:42:52.922087 22479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260809 04:42:52.922327 22510 tablet_bootstrap.cc:492] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Bootstrap starting.
I20260809 04:42:52.928676 22510 log.cc:824] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:52.932324 22479 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:43109
I20260809 04:42:52.932390 22629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:43109 every 8 connection(s)
I20260809 04:42:52.933442 22479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:42:52.937038 22510 tablet_bootstrap.cc:492] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:42:52.937520 22510 tablet_bootstrap.cc:492] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Bootstrap complete.
I20260809 04:42:52.937882 22510 ts_tablet_manager.cc:1397] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Time spent bootstrapping tablet: real 0.016s user 0.015s sys 0.000s
I20260809 04:42:52.940022 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 22479
I20260809 04:42:52.940418 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:33693
--local_ip_for_outbound_sockets=127.5.222.195
--tserver_master_addrs=127.5.222.254:38293
--webserver_port=45607
--webserver_interface=127.5.222.195
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:42:52.940656 22510 raft_consensus.cc:348] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.941181 22510 raft_consensus.cc:729] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2bc1810f85947818231f5d89ee41ed4, State: Initialized, Role: FOLLOWER
I20260809 04:42:52.941676 22510 consensus_queue.cc:260] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } }
I20260809 04:42:52.942745 22510 ts_tablet_manager.cc:1428] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
Aug 09 04:42:52 dist-test-slave-vxj2 krb5kdc[21659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:52.948559 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.935440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:58879 (local address 127.5.222.254:38293)
0809 04:42:52.935607 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:52.935612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:52.936224 (+ 612us) server_negotiation.cc:407] Connection header received
0809 04:42:52.937404 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:52.937409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:52.937483 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:52.937595 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:52.939104 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.939638 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.941323 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.941660 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.941789 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.944871 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:52.944874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:52.944877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:52.944909 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:52.946481 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:52.947007 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:52.947012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:52.947014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:52.947098 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:52.947467 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:52.947470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:52.947472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:52.947656 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:52.947765 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:52.948246 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 04:42:52.948384 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:42:52.949379 22633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.935557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:58879)
0809 04:42:52.936047 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:52.936098 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:52.937142 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:52.937836 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:52.937844 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:52.938334 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:52.938939 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.938956 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.939794 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.939798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.941098 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.941110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.941885 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.941893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.942140 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.943103 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:52.943119 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:52.944690 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:52.946624 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:52.946629 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:52.946642 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:52.946896 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:52.947202 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:52.947205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:52.947207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:52.947341 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:52.947763 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:52.947779 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:52.948016 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 04:42:52.948202 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 04:42:52.948518 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":115,"threads_started":1}
I20260809 04:42:52.950613 22631 heartbeater.cc:347] Connected to a master server at 127.5.222.254:38293
I20260809 04:42:52.950850 22631 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:52.951339 22631 heartbeater.cc:511] Master 127.5.222.254:38293 requested a full tablet report, sending...
I20260809 04:42:52.953032 22277 ts_manager.cc:194] Registered new tserver with Master: c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109)
I20260809 04:42:52.954357 22277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:58879
I20260809 04:42:52.956563 22631 heartbeater.cc:503] Master 127.5.222.254:38293 was elected leader, sending a full tablet report...
Aug 09 04:42:52 dist-test-slave-vxj2 krb5kdc[21659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.194@KRBTEST.COM
I20260809 04:42:52.963924 22482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.954037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43109 (local address 127.0.0.1:45402)
0809 04:42:52.954218 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:52.954236 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:52.954315 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:52.955029 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:52.955032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:52.955249 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:52.955352 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.955359 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.956626 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.956629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.957544 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:52.957554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.958259 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.958265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:52.958362 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.958826 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:52.958848 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:52.960181 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:52.962317 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:52.962320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:52.962322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:52.962527 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:52.962833 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:52.962837 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:52.962858 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:52.962953 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:52.963384 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:52.963387 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:52.963492 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 04:42:52.963612 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 04:42:52.963755 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 04:42:52.964033 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:52.954121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45402 (local address 127.5.222.194:43109)
0809 04:42:52.954581 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:52.954594 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:52.954617 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:42:52.954684 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:52.954689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:52.954870 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:52.955025 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:52.955480 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.956504 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.957674 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:52.958143 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:52.958227 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:52.960320 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:52.960332 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:52.960350 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:52.960371 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:52.962202 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:52.962649 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:52.962654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:52.962655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:52.962735 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:52.963046 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:52.963051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:52.963053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:52.963273 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:52.963415 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:52.963693 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 04:42:52.963843 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":355,"thread_start_us":126,"threads_started":1}
I20260809 04:42:52.964916 22557 tablet_service.cc:1514] Processing DeleteTablet for tablet 9754972977234caf87df45f68b11b0dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:42:52 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:45402
I20260809 04:42:52.966022 22637 tablet_replica.cc:330] stopping tablet replica
I20260809 04:42:52.966325 22637 raft_consensus.cc:2229] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:42:52.966588 22637 raft_consensus.cc:2258] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:42:52.967549 22637 ts_tablet_manager.cc:1905] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:42:52.971288 22637 ts_tablet_manager.cc:1918] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:42:52.971427 22637 log.cc:1196] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/9754972977234caf87df45f68b11b0dc
I20260809 04:42:52.971721 22637 ts_tablet_manager.cc:1939] T 9754972977234caf87df45f68b11b0dc P c2bc1810f85947818231f5d89ee41ed4: Deleting consensus metadata
I20260809 04:42:52.973143 22260 catalog_manager.cc:4841] TS c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109): tablet 9754972977234caf87df45f68b11b0dc (table test-table [id=beafb62b89ba4aa0b1c500fa99974d2d]) successfully deleted
W20260809 04:42:53.056155 22634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:53.056396 22634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:53.056465 22634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:53.060371 22634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:53.060422 22634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:53.060527 22634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:42:53.066077 22634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:33693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=45607
--tserver_master_addrs=127.5.222.254:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:53.067056 22634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:53.068089 22634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:42:53.077512 22634 server_base.cc:1034] running on GCE node
W20260809 04:42:53.077491 22643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:53.077494 22642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:53.077807 22645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:53.078138 22634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:53.078857 22634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:53.080009 22634 hybrid_clock.cc:648] HybridClock initialized: now 1786250573079994 us; error 26 us; skew 500 ppm
Aug 09 04:42:53 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250573, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:53.082762 22634 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:42:53.083647 22634 webserver.cc:460] Webserver started at http://127.5.222.195:45607/ using document root <none> and password file <none>
I20260809 04:42:53.084205 22634 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:53.084276 22634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:53.087993 22634 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:42:53.089987 22653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:53.091243 22634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:42:53.091361 22634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "e6324db4e82b426fb0a26b7178377638"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "1db1be8d871f2fa4c86d16b7992f0859"
server_key_iv: "2b5db305f5b44952e38d0ea568a77ad8"
server_key_version: "encryptionkey@0"
I20260809 04:42:53.091910 22634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:53.118429 22634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:53.121341 22634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:53.121516 22634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:53.122049 22634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:53.122816 22634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:42:53.122864 22634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:53.122903 22634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:42:53.122918 22634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:53.132014 22634 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:33693
I20260809 04:42:53.132042 22772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:33693 every 8 connection(s)
I20260809 04:42:53.132985 22634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:42:53.138393 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 22634
Aug 09 04:42:53 dist-test-slave-vxj2 krb5kdc[21659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250573, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:42:53.147549 22777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.139096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37572 (local address 127.5.222.254:38293)
0809 04:42:53.139390 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:53.139393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:53.139601 (+ 208us) server_negotiation.cc:407] Connection header received
0809 04:42:53.139720 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:53.139722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:53.139759 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:53.139833 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:53.140727 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.141278 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.142314 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.142533 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.142661 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:53.143993 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:53.143997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:53.144000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:53.144040 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:53.145636 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:53.146060 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:53.146064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:53.146066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:53.146178 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:53.146489 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:53.146493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:53.146495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:53.146734 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:53.146937 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:53.147143 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 04:42:53.147270 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":89,"threads_started":1}
I20260809 04:42:53.147864 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.134780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:54937 (local address 127.5.222.254:38293)
0809 04:42:53.134932 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:53.134936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:53.135764 (+ 828us) server_negotiation.cc:407] Connection header received
0809 04:42:53.137174 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:53.137178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:53.137239 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:53.137346 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:53.138509 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.139130 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.140869 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.141088 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.141235 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:53.143665 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:53.143674 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:53.143676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:53.143699 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:53.145538 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:53.146057 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:53.146061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:53.146063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:53.146155 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:53.146513 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:53.146518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:53.146519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:53.146785 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:53.146897 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:53.147425 (+ 528us) server_negotiation.cc:299] Negotiation successful
0809 04:42:53.147541 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:42:53.148633 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.135133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:54937)
0809 04:42:53.135598 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:53.135639 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:53.136925 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:53.137513 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:53.137520 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:53.137872 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:53.138361 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:53.138373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.139262 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.139269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:53.140737 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:53.140760 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.141252 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.141256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:53.141432 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:53.142064 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:53.142079 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:53.143497 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:53.145670 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:53.145676 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:53.145690 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:53.145948 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:53.146264 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:53.146267 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:53.146269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:53.146404 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:53.146897 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:53.146905 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:53.147140 (+ 235us) client_negotiation.cc:769] Sending connection context
0809 04:42:53.147379 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 04:42:53.147727 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":115,"threads_started":1}
I20260809 04:42:53.149833 22773 heartbeater.cc:347] Connected to a master server at 127.5.222.254:38293
I20260809 04:42:53.150045 22773 heartbeater.cc:464] Registering TS with master...
I20260809 04:42:53.150480 22773 heartbeater.cc:511] Master 127.5.222.254:38293 requested a full tablet report, sending...
I20260809 04:42:53.151397 22277 ts_manager.cc:194] Registered new tserver with Master: e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693)
I20260809 04:42:53.151942 22277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:54937
I20260809 04:42:53.161262 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:42:53.165656 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.161946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37576 (local address 127.5.222.254:38293)
0809 04:42:53.162067 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:53.162071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:53.162242 (+ 171us) server_negotiation.cc:407] Connection header received
0809 04:42:53.162454 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:53.162457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:53.162555 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:53.162641 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:42:53.162857 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.163350 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.164268 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.164467 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.164534 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:53.164855 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:42:53.165328 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:42:53.165421 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:53.165509 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 04:42:53.165556 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:42:53.166256 22277 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:37576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:42:53.169462 22277 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:42:53.177275 22557 tablet_service.cc:1467] Processing CreateTablet for tablet bd395a54b9354f8c815a0e5410e79766 (DEFAULT_TABLE table=test-table [id=8b111ef7687e42ffa18eb46d71a0eab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:42:53.177275 22405 tablet_service.cc:1467] Processing CreateTablet for tablet bd395a54b9354f8c815a0e5410e79766 (DEFAULT_TABLE table=test-table [id=8b111ef7687e42ffa18eb46d71a0eab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:42:53.177901 22405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd395a54b9354f8c815a0e5410e79766. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:53.177901 22557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd395a54b9354f8c815a0e5410e79766. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:53.180868 22362 tablet_bootstrap.cc:492] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: Bootstrap starting.
I20260809 04:42:53.180963 22510 tablet_bootstrap.cc:492] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4: Bootstrap starting.
I20260809 04:42:53.181874 22362 tablet_bootstrap.cc:654] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:53.182045 22510 tablet_bootstrap.cc:654] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:53.182116 22482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.176410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:33693 (local address 127.0.0.1:47506)
0809 04:42:53.176599 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:53.176616 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:53.176699 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:53.177486 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:53.177489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:53.177508 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:53.177606 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:53.177616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.179043 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.179049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:53.180015 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:53.180025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.181588 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.181592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:53.181692 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:53.181753 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 04:42:53.181860 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 04:42:53.181951 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260809 04:42:53.182333 22782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.176646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47506 (local address 127.5.222.195:33693)
0809 04:42:53.177016 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:53.177026 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:53.177046 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:42:53.177108 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:53.177112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:53.177330 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:53.177503 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:53.177738 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.178901 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.180144 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.181414 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.181565 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:53.181712 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:53.181903 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 04:42:53.182177 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":167,"threads_started":1}
I20260809 04:42:53.183573 22510 tablet_bootstrap.cc:492] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4: No bootstrap required, opened a new log
I20260809 04:42:53.183650 22510 ts_tablet_manager.cc:1397] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:42:53.184075 22510 raft_consensus.cc:348] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:53.184201 22510 raft_consensus.cc:374] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:53.184217 22362 tablet_bootstrap.cc:492] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: No bootstrap required, opened a new log
I20260809 04:42:53.184239 22510 raft_consensus.cc:729] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2bc1810f85947818231f5d89ee41ed4, State: Initialized, Role: FOLLOWER
I20260809 04:42:53.184309 22362 ts_tablet_manager.cc:1397] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:42:53.184376 22510 consensus_queue.cc:260] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:53.184623 22510 ts_tablet_manager.cc:1428] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260809 04:42:53.184851 22632 tablet.cc:2369] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:42:53.184834 22362 raft_consensus.cc:348] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:53.184970 22362 raft_consensus.cc:374] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:53.185029 22362 raft_consensus.cc:729] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339f6af8d9a344918e391a084ddd0ab3, State: Initialized, Role: FOLLOWER
I20260809 04:42:53.185200 22362 consensus_queue.cc:260] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:53.185529 22362 ts_tablet_manager.cc:1428] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 04:42:53.185868 22705 tablet_service.cc:1467] Processing CreateTablet for tablet bd395a54b9354f8c815a0e5410e79766 (DEFAULT_TABLE table=test-table [id=8b111ef7687e42ffa18eb46d71a0eab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:42:53.187047 22705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd395a54b9354f8c815a0e5410e79766. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:42:53.193783 22785 tablet_bootstrap.cc:492] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638: Bootstrap starting.
I20260809 04:42:53.196668 22785 tablet_bootstrap.cc:654] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638: Neither blocks nor log segments found. Creating new log.
I20260809 04:42:53.197492 22785 log.cc:824] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:53.199234 22785 tablet_bootstrap.cc:492] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638: No bootstrap required, opened a new log
I20260809 04:42:53.199442 22785 ts_tablet_manager.cc:1397] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:42:53.202589 22785 raft_consensus.cc:348] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:53.202843 22785 raft_consensus.cc:374] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:42:53.202899 22785 raft_consensus.cc:729] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6324db4e82b426fb0a26b7178377638, State: Initialized, Role: FOLLOWER
I20260809 04:42:53.203306 22785 consensus_queue.cc:260] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:53.203944 22773 heartbeater.cc:503] Master 127.5.222.254:38293 was elected leader, sending a full tablet report...
I20260809 04:42:53.204254 22785 ts_tablet_manager.cc:1428] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 04:42:53.210161 22481 raft_consensus.cc:482] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:42:53.210317 22481 raft_consensus.cc:504] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:53.211314 22481 leader_election.cc:288] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693), c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109)
I20260809 04:42:53.214941 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.211495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:33693 (local address 127.5.222.193:49005)
0809 04:42:53.211603 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:53.211622 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:53.211718 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:53.212007 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:53.212010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:53.212027 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:53.212139 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:53.212144 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.212861 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.212865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:53.213785 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:53.213795 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.214492 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.214495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:53.214610 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:53.214644 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 04:42:53.214739 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 04:42:53.214816 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":36}
I20260809 04:42:53.214941 22782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.211577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:49005 (local address 127.5.222.195:33693)
0809 04:42:53.211710 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:53.211715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:53.211730 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:42:53.211871 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:53.211874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:53.211910 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:53.211995 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:53.212244 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.212737 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.213941 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.214372 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.214501 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:53.214549 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:53.214756 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 04:42:53.214820 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:42:53.215170 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.211751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:40855 (local address 127.5.222.194:43109)
0809 04:42:53.211877 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:53.211882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:53.211960 (+ 78us) server_negotiation.cc:407] Connection header received
0809 04:42:53.212122 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:53.212124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:53.212174 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:53.212274 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:42:53.212559 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.213087 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.214137 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.214571 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.214707 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:53.214844 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:53.214940 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 04:42:53.215007 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:42:53.215504 22788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.211577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43109 (local address 127.5.222.193:40855)
0809 04:42:53.211888 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:53.211902 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:53.212005 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:53.212284 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:53.212287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:53.212302 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:42:53.212444 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:53.212450 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.213216 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.213222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:53.214010 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:53.214019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.214707 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.214711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:53.214810 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:53.214830 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:42:53.214914 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:42:53.214986 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":1,"thread_start_us":66,"threads_started":1}
I20260809 04:42:53.215577 22726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd395a54b9354f8c815a0e5410e79766" candidate_uuid: "339f6af8d9a344918e391a084ddd0ab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6324db4e82b426fb0a26b7178377638" is_pre_election: true
I20260809 04:42:53.215855 22726 raft_consensus.cc:2454] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 339f6af8d9a344918e391a084ddd0ab3 in term 0.
I20260809 04:42:53.216070 22580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd395a54b9354f8c815a0e5410e79766" candidate_uuid: "339f6af8d9a344918e391a084ddd0ab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2bc1810f85947818231f5d89ee41ed4" is_pre_election: true
I20260809 04:42:53.216334 22580 raft_consensus.cc:2454] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 339f6af8d9a344918e391a084ddd0ab3 in term 0.
I20260809 04:42:53.216359 22357 leader_election.cc:302] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339f6af8d9a344918e391a084ddd0ab3, e6324db4e82b426fb0a26b7178377638; no voters:
I20260809 04:42:53.216650 22481 raft_consensus.cc:2781] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:42:53.216742 22481 raft_consensus.cc:482] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:42:53.216792 22481 raft_consensus.cc:3037] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:53.217911 22481 raft_consensus.cc:504] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:53.218240 22481 leader_election.cc:288] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 1 election: Requested vote from peers e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693), c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109)
I20260809 04:42:53.218556 22726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd395a54b9354f8c815a0e5410e79766" candidate_uuid: "339f6af8d9a344918e391a084ddd0ab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6324db4e82b426fb0a26b7178377638"
I20260809 04:42:53.218632 22580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd395a54b9354f8c815a0e5410e79766" candidate_uuid: "339f6af8d9a344918e391a084ddd0ab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2bc1810f85947818231f5d89ee41ed4"
I20260809 04:42:53.218706 22726 raft_consensus.cc:3037] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:53.218744 22580 raft_consensus.cc:3037] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:42:53.219955 22580 raft_consensus.cc:2454] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 339f6af8d9a344918e391a084ddd0ab3 in term 1.
I20260809 04:42:53.219966 22726 raft_consensus.cc:2454] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 339f6af8d9a344918e391a084ddd0ab3 in term 1.
I20260809 04:42:53.220301 22357 leader_election.cc:302] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339f6af8d9a344918e391a084ddd0ab3, e6324db4e82b426fb0a26b7178377638; no voters:
I20260809 04:42:53.220559 22481 raft_consensus.cc:2781] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:42:53.220878 22481 raft_consensus.cc:686] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 LEADER]: Becoming Leader. State: Replica: 339f6af8d9a344918e391a084ddd0ab3, State: Running, Role: LEADER
I20260809 04:42:53.221199 22481 consensus_queue.cc:237] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:53.223999 22277 catalog_manager.cc:5495] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 339f6af8d9a344918e391a084ddd0ab3 (127.5.222.193). New cstate: current_term: 1 leader_uuid: "339f6af8d9a344918e391a084ddd0ab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } health_report { overall_health: UNKNOWN } } }
I20260809 04:42:53.235522 22483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.231916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.5.222.193:41333)
0809 04:42:53.232038 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:53.232044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:53.232077 (+ 33us) server_negotiation.cc:407] Connection header received
0809 04:42:53.232268 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:53.232271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:53.232322 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:53.232432 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:42:53.232676 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.233265 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.234101 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:53.234338 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:53.234450 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:53.234723 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:42:53.235134 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:42:53.235260 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:53.235324 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 04:42:53.235377 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 04:42:53.238933 22476 tablet.cc:2369] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:42:53.239491 22726 raft_consensus.cc:1261] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638 [term 1 FOLLOWER]: Refusing update from remote peer 339f6af8d9a344918e391a084ddd0ab3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:42:53.239504 22580 raft_consensus.cc:1261] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4 [term 1 FOLLOWER]: Refusing update from remote peer 339f6af8d9a344918e391a084ddd0ab3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:42:53.240231 22481 consensus_queue.cc:1035] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:42:53.240444 22789 consensus_queue.cc:1035] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:42:53.247093 22793 mvcc.cc:205] Tried to move back new op lower bound from 7316482347982516224 to 7316482347916746752. Current Snapshot: MvccSnapshot[applied={T|T < 7316482347982516224}]
I20260809 04:42:53.248682 22792 mvcc.cc:205] Tried to move back new op lower bound from 7316482347982516224 to 7316482347916746752. Current Snapshot: MvccSnapshot[applied={T|T < 7316482347982516224 or (T in {7316482347982516224})}]
I20260809 04:42:53.249635 22795 mvcc.cc:205] Tried to move back new op lower bound from 7316482347982516224 to 7316482347916746752. Current Snapshot: MvccSnapshot[applied={T|T < 7316482347982516224}]
I20260809 04:42:53.254623 22277 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:37576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:42:53.254796 22277 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:37576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:42:53.257292 22277 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Sending DeleteTablet for 3 replicas of tablet bd395a54b9354f8c815a0e5410e79766
I20260809 04:42:53.258061 22705 tablet_service.cc:1514] Processing DeleteTablet for tablet bd395a54b9354f8c815a0e5410e79766 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:42:53 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:47506
I20260809 04:42:53.258121 22557 tablet_service.cc:1514] Processing DeleteTablet for tablet bd395a54b9354f8c815a0e5410e79766 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:42:53 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:45402
I20260809 04:42:53.258339 22405 tablet_service.cc:1514] Processing DeleteTablet for tablet bd395a54b9354f8c815a0e5410e79766 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:42:53 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:51642
I20260809 04:42:53.258558 22637 tablet_replica.cc:330] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4: stopping tablet replica
I20260809 04:42:53.258682 22803 tablet_replica.cc:330] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638: stopping tablet replica
I20260809 04:42:53.258776 22637 raft_consensus.cc:2229] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:42:53.258859 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 22317
I20260809 04:42:53.258942 22637 raft_consensus.cc:2258] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:42:53.259148 22484 tablet_replica.cc:330] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: stopping tablet replica
I20260809 04:42:53.259258 22803 raft_consensus.cc:2229] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:42:53.259300 22484 raft_consensus.cc:2229] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:42:53.261140 22803 raft_consensus.cc:2258] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:42:53.263142 22803 ts_tablet_manager.cc:1905] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:42:53.264532 22637 ts_tablet_manager.cc:1905] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:42:53.265609 22803 ts_tablet_manager.cc:1918] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:42:53.265733 22803 log.cc:1196] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/bd395a54b9354f8c815a0e5410e79766
I20260809 04:42:53.266077 22803 ts_tablet_manager.cc:1939] T bd395a54b9354f8c815a0e5410e79766 P e6324db4e82b426fb0a26b7178377638: Deleting consensus metadata
I20260809 04:42:53.267258 22259 catalog_manager.cc:4841] TS e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693): tablet bd395a54b9354f8c815a0e5410e79766 (table test-table [id=8b111ef7687e42ffa18eb46d71a0eab4]) successfully deleted
I20260809 04:42:53.267719 22637 ts_tablet_manager.cc:1918] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:42:53.267813 22637 log.cc:1196] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/bd395a54b9354f8c815a0e5410e79766
I20260809 04:42:53.268077 22637 ts_tablet_manager.cc:1939] T bd395a54b9354f8c815a0e5410e79766 P c2bc1810f85947818231f5d89ee41ed4: Deleting consensus metadata
I20260809 04:42:53.268821 22260 catalog_manager.cc:4841] TS c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109): tablet bd395a54b9354f8c815a0e5410e79766 (table test-table [id=8b111ef7687e42ffa18eb46d71a0eab4]) successfully deleted
W20260809 04:42:53.270167 22260 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:41333 (error 108)
I20260809 04:42:53.270427 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 22479
I20260809 04:42:53.271799 22482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.271515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:41333 (local address 127.0.0.1:51654)
0809 04:42:53.271648 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:53.271728 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:41333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 04:42:53.271975 22260 catalog_manager.cc:4584] TS 339f6af8d9a344918e391a084ddd0ab3 (127.5.222.193:41333): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd395a54b9354f8c815a0e5410e79766: Network error: Client connection negotiation failed: client connection to 127.5.222.193:41333: connect: Connection refused (error 111)
I20260809 04:42:53.277959 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 22634
I20260809 04:42:53.284817 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 22241
I20260809 04:42:53.293113 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:38293
--webserver_interface=127.5.222.254
--webserver_port=40485
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:38293
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:42:53.409965 22805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:53.410202 22805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:53.410238 22805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:53.414165 22805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:42:53.414237 22805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:53.414256 22805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:53.414273 22805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:42:53.414288 22805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:42:53.419440 22805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:38293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:38293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:53.420442 22805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:53.421489 22805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:53.429940 22814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:53.429940 22810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:53.430016 22805 server_base.cc:1034] running on GCE node
W20260809 04:42:53.429993 22811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:53.430830 22805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:53.431885 22805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:53.433048 22805 hybrid_clock.cc:648] HybridClock initialized: now 1786250573433028 us; error 32 us; skew 500 ppm
Aug 09 04:42:53 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250573, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:53.435786 22805 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:42:53.436673 22805 webserver.cc:460] Webserver started at http://127.5.222.254:40485/ using document root <none> and password file <none>
I20260809 04:42:53.437279 22805 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:53.437371 22805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:53.441071 22805 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:42:53.443239 22821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:53.444442 22805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:42:53.444585 22805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "3f00aa306161467cac4c773f9995371f"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "402182a0a87db3802d66aadfd52a3448"
server_key_iv: "2d732c6377194ba1da05dfa3a5b67540"
server_key_version: "encryptionkey@0"
I20260809 04:42:53.445127 22805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:53.461565 22805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:53.464474 22805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:53.464677 22805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:53.472231 22805 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:38293
I20260809 04:42:53.472540 22876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:38293 every 8 connection(s)
I20260809 04:42:53.473371 22805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:42:53.477999 22877 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:42:53.479858 22877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Bootstrap starting.
I20260809 04:42:53.480008 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 22805
I20260809 04:42:53.480576 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:41333
--local_ip_for_outbound_sockets=127.5.222.193
--tserver_master_addrs=127.5.222.254:38293
--webserver_port=46189
--webserver_interface=127.5.222.193
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:42:53.499907 22877 log.cc:824] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:53.512337 22877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:42:53.512825 22877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f: Bootstrap complete.
I20260809 04:42:53.517607 22877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } }
I20260809 04:42:53.518055 22877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f00aa306161467cac4c773f9995371f, State: Initialized, Role: FOLLOWER
I20260809 04:42:53.518626 22877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } }
I20260809 04:42:53.518759 22877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:42:53.518851 22877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:42:53.518987 22877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 2 FOLLOWER]: Advancing to term 3
I20260809 04:42:53.521034 22877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } }
I20260809 04:42:53.521432 22877 leader_election.cc:302] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f00aa306161467cac4c773f9995371f; no voters:
I20260809 04:42:53.521806 22877 leader_election.cc:288] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 04:42:53.521907 22881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 3 FOLLOWER]: Leader election won for term 3
I20260809 04:42:53.522135 22881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [term 3 LEADER]: Becoming Leader. State: Replica: 3f00aa306161467cac4c773f9995371f, State: Running, Role: LEADER
I20260809 04:42:53.522464 22881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } }
I20260809 04:42:53.523180 22877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:42:53.523487 22883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f00aa306161467cac4c773f9995371f. Latest consensus state: current_term: 3 leader_uuid: "3f00aa306161467cac4c773f9995371f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } } }
I20260809 04:42:53.523627 22883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:53.523890 22882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3f00aa306161467cac4c773f9995371f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f00aa306161467cac4c773f9995371f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38293 } } }
I20260809 04:42:53.524004 22882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f00aa306161467cac4c773f9995371f [sys.catalog]: This master's current role is: LEADER
I20260809 04:42:53.524353 22885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:42:53.530326 22885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:42:53.531215 22885 catalog_manager.cc:1259] Loaded cluster ID: 267cc2e146b94e8e8dcf367a521314a8
I20260809 04:42:53.531284 22885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 04:42:53.532506 22885 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f26591fb9ad google::LogMessage::Fail() at ??:0
@ 0x7f26591fe88a google::LogMessage::SendToLog() at ??:0
@ 0x7f26591fb4de google::LogMessage::Flush() at ??:0
@ 0x7f26591fca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2663b3afdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f2663b3bd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2663b38579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f2663b7c640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f265b44d3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f265b44dc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f265b44f3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f265b43e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f265ca826db start_thread at ??:0
@ 0x7f2657cf961f clone at ??:0
W20260809 04:42:53.623739 22879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:53.624147 22879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:53.624236 22879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:53.630771 22879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:53.630860 22879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:53.630956 22879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:42:53.639276 22879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:41333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=46189
--tserver_master_addrs=127.5.222.254:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:53.640719 22879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:53.642102 22879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:53.654623 22920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:53.654611 22919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:53.654611 22922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:53.655074 22879 server_base.cc:1034] running on GCE node
I20260809 04:42:53.655481 22879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:53.656157 22879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:53.657348 22879 hybrid_clock.cc:648] HybridClock initialized: now 1786250573657327 us; error 37 us; skew 500 ppm
Aug 09 04:42:53 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250573, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:53.660238 22879 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:42:53.661234 22879 webserver.cc:460] Webserver started at http://127.5.222.193:46189/ using document root <none> and password file <none>
I20260809 04:42:53.661801 22879 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:53.661873 22879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:53.665793 22879 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:42:53.668258 22931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:53.669747 22879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:42:53.669904 22879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "339f6af8d9a344918e391a084ddd0ab3"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "b796548ca7f6b010e272dd932b9b97cc"
server_key_iv: "915250fd03606a47a825734bbfd235fd"
server_key_version: "encryptionkey@0"
I20260809 04:42:53.670501 22879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:53.691040 22879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:53.694518 22879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:53.694751 22879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:53.695374 22879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:53.696516 22938 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 04:42:53.702064 22879 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 04:42:53.702167 22879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 04:42:53.702258 22879 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 04:42:53.704116 22879 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 04:42:53.704185 22879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 04:42:53.704504 22938 tablet_bootstrap.cc:492] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: Bootstrap starting.
I20260809 04:42:53.715606 22879 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:41333
I20260809 04:42:53.716624 22879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:42:53.717543 23052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:41333 every 8 connection(s)
I20260809 04:42:53.718570 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 22879
I20260809 04:42:53.719033 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:43109
--local_ip_for_outbound_sockets=127.5.222.194
--tserver_master_addrs=127.5.222.254:38293
--webserver_port=44739
--webserver_interface=127.5.222.194
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:42:53.726703 22938 log.cc:824] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: Log is configured to *not* fsync() on all Append() calls
I20260809 04:42:53.735787 22938 tablet_bootstrap.cc:492] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:42:53.736276 22938 tablet_bootstrap.cc:492] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: Bootstrap complete.
I20260809 04:42:53.737743 22938 ts_tablet_manager.cc:1397] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: Time spent bootstrapping tablet: real 0.033s user 0.025s sys 0.007s
I20260809 04:42:53.741120 22938 raft_consensus.cc:348] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:53.741690 22938 raft_consensus.cc:729] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339f6af8d9a344918e391a084ddd0ab3, State: Initialized, Role: FOLLOWER
I20260809 04:42:53.742237 22938 consensus_queue.cc:260] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:53.743359 22938 ts_tablet_manager.cc:1428] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260809 04:42:53.773661 23055 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:42:53.719023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:51679)
0809 04:42:53.720410 (+ 1387us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:53.720474 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:53.721565 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:53.772645 (+ 51080us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1137,"thread_start_us":162,"threads_started":1}
W20260809 04:42:53.774294 23053 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:38293 (0 consecutive failures): Network error: Failed to ping master at 127.5.222.254:38293: Client connection negotiation failed: client connection to 127.5.222.254:38293: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 04:42:53.775077 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.774783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:57241)
0809 04:42:53.774924 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:53.775007 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 04:42:53.775816 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.775587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:32791)
0809 04:42:53.775701 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:53.775752 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 04:42:53.776003 23053 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:42:53.836268 23056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:53.836513 23056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:53.836598 23056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:53.840631 23056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:53.840695 23056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:53.840803 23056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:42:53.845994 23056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:43109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=44739
--tserver_master_addrs=127.5.222.254:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:53.847030 23056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:53.848029 23056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:53.857622 23064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:53.857643 23065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:53.857632 23067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:53.858105 23056 server_base.cc:1034] running on GCE node
I20260809 04:42:53.858502 23056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:53.859117 23056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:53.860301 23056 hybrid_clock.cc:648] HybridClock initialized: now 1786250573860268 us; error 44 us; skew 500 ppm
Aug 09 04:42:53 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250573, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:53.863260 23056 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:42:53.864104 23056 webserver.cc:460] Webserver started at http://127.5.222.194:44739/ using document root <none> and password file <none>
I20260809 04:42:53.864658 23056 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:53.864753 23056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:53.868713 23056 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:42:53.870823 23074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:53.872071 23056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:42:53.872215 23056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "c2bc1810f85947818231f5d89ee41ed4"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "6f308f4a04159b6ed4b76a847bed7cec"
server_key_iv: "e0eabbc9f540b269c648f88ea026ce6e"
server_key_version: "encryptionkey@0"
I20260809 04:42:53.872763 23056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:53.899284 23056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:53.902376 23056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:53.902585 23056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:53.903203 23056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:53.904042 23056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:42:53.904116 23056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:53.904189 23056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:42:53.904242 23056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:53.913520 23056 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:43109
I20260809 04:42:53.913549 23192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:43109 every 8 connection(s)
I20260809 04:42:53.914522 23056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:42:53.917232 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 23056
I20260809 04:42:53.917972 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:33693
--local_ip_for_outbound_sockets=127.5.222.195
--tserver_master_addrs=127.5.222.254:38293
--webserver_port=45607
--webserver_interface=127.5.222.195
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:42:53.918470 23195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.916589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:45651)
0809 04:42:53.917057 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:53.917642 (+ 585us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":105,"threads_started":1}
W20260809 04:42:53.919049 23193 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:38293 (0 consecutive failures): Network error: Failed to ping master at 127.5.222.254:38293: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
I20260809 04:42:53.919690 23195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.919461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:56303)
0809 04:42:53.919575 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:53.919628 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 04:42:53.920396 23195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:53.920193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:43015)
0809 04:42:53.920296 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:53.920348 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 04:42:53.920591 23193 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:42:54.030411 23196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:42:54.030705 23196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:42:54.030776 23196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:42:54.034870 23196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:42:54.034950 23196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:42:54.035036 23196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:42:54.040251 23196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:33693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=45607
--tserver_master_addrs=127.5.222.254:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:42:54.041320 23196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:42:54.042364 23196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:42:54.052325 23201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:54.052340 23202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:42:54.052372 23204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:42:54.052405 23196 server_base.cc:1034] running on GCE node
I20260809 04:42:54.052969 23196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:42:54.053709 23196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:42:54.054896 23196 hybrid_clock.cc:648] HybridClock initialized: now 1786250574054874 us; error 47 us; skew 500 ppm
Aug 09 04:42:54 dist-test-slave-vxj2 krb5kdc[21659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250574, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:42:54.057860 23196 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:42:54.058744 23196 webserver.cc:460] Webserver started at http://127.5.222.195:45607/ using document root <none> and password file <none>
I20260809 04:42:54.059303 23196 fs_manager.cc:359] Metadata directory not provided
I20260809 04:42:54.059366 23196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:42:54.063234 23196 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:42:54.065315 23212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:54.066614 23196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:42:54.066759 23196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "e6324db4e82b426fb0a26b7178377638"
format_stamp: "Formatted at 2026-08-09 04:42:51 on dist-test-slave-vxj2"
server_key: "1db1be8d871f2fa4c86d16b7992f0859"
server_key_iv: "2b5db305f5b44952e38d0ea568a77ad8"
server_key_version: "encryptionkey@0"
I20260809 04:42:54.067297 23196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:42:54.093968 23196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:42:54.096967 23196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:42:54.097198 23196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:42:54.097831 23196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:42:54.098770 23196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:42:54.098842 23196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:54.098914 23196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:42:54.098965 23196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:42:54.108059 23196 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:33693
I20260809 04:42:54.108089 23331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:33693 every 8 connection(s)
I20260809 04:42:54.109047 23196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:42:54.112776 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:54.111189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:53863)
0809 04:42:54.111652 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:54.112082 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":116,"threads_started":1}
W20260809 04:42:54.113339 23332 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:38293 (0 consecutive failures): Network error: Failed to ping master at 127.5.222.254:38293: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
I20260809 04:42:54.113911 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:54.113705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:43833)
0809 04:42:54.113815 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:54.113860 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 04:42:54.114488 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:54.114301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:43549)
0809 04:42:54.114400 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:54.114440 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 04:42:54.114684 23332 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 04:42:54.115588 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 23196
I20260809 04:42:54.777035 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:54.776545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:49749)
0809 04:42:54.776852 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:54.776915 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":73,"threads_started":1}
I20260809 04:42:54.921643 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:54.921227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:43155)
0809 04:42:54.921481 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:54.921539 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":83,"threads_started":1}
I20260809 04:42:55.094331 23339 raft_consensus.cc:482] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:42:55.094575 23339 raft_consensus.cc:504] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:55.095636 23339 leader_election.cc:288] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693), c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109)
Aug 09 04:42:55 dist-test-slave-vxj2 krb5kdc[21659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250573, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.195@KRBTEST.COM
Aug 09 04:42:55 dist-test-slave-vxj2 krb5kdc[21659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250573, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.194@KRBTEST.COM
I20260809 04:42:55.113085 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:55.095877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:33693 (local address 127.5.222.193:55199)
0809 04:42:55.096078 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:55.096095 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:55.096203 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:55.098445 (+ 2242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:55.098452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:55.099240 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:55.100040 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:55.100063 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:55.102032 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:55.102036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:55.103734 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:55.103746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:55.104723 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:55.104727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:55.105007 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:55.105845 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:55.105856 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:55.107934 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:55.110666 (+ 2732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:55.110669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:55.110671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:55.111000 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:55.111376 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:55.111379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:55.111381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:55.111547 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:55.112193 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:55.112199 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:55.112411 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 04:42:55.112583 (+ 172us) client_negotiation.cc:240] Negotiation successful
0809 04:42:55.112889 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":436,"spinlock_wait_cycles":7168}
I20260809 04:42:55.113210 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:55.096058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:55199 (local address 127.5.222.195:33693)
0809 04:42:55.096495 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:55.096512 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:55.096559 (+ 47us) server_negotiation.cc:407] Connection header received
0809 04:42:55.097507 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:55.097516 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:55.097955 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:55.098290 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:55.100254 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:55.101852 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:55.103869 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:55.104588 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:55.104767 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:55.108147 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:55.108152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:55.108164 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:55.108224 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:55.110533 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:55.111119 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:55.111125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:55.111130 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:55.111268 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:55.111660 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:55.111664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:55.111665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:55.112054 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:55.112198 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:55.112708 (+ 510us) server_negotiation.cc:299] Negotiation successful
0809 04:42:55.112999 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":86,"threads_started":1}
I20260809 04:42:55.113270 23341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:55.096050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:58931 (local address 127.5.222.194:43109)
0809 04:42:55.096510 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 04:42:55.096530 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:42:55.096560 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:42:55.097507 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:42:55.097516 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:42:55.097955 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:42:55.098290 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:42:55.100254 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:55.101852 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:55.103869 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:42:55.104588 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:42:55.104771 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:55.108331 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:42:55.108339 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:42:55.108354 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:42:55.108385 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:42:55.110518 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:55.111205 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:55.111210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:55.111215 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:55.111370 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:42:55.111640 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:42:55.111643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:42:55.111644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:42:55.112113 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:42:55.112222 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:42:55.112858 (+ 636us) server_negotiation.cc:299] Negotiation successful
0809 04:42:55.113070 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":85,"threads_started":1}
I20260809 04:42:55.114078 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:55.095877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43109 (local address 127.5.222.193:58931)
0809 04:42:55.096191 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:55.096203 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:42:55.096280 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:42:55.098445 (+ 2165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:42:55.098452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:42:55.099240 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:42:55.100040 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:55.100063 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:55.102036 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:55.102038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:55.103734 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:42:55.103746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:42:55.104723 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:42:55.104727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:42:55.105010 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:42:55.105814 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:42:55.105836 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:42:55.108128 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:42:55.110646 (+ 2518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:55.110652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:55.110664 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:55.111056 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:55.111463 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:42:55.111466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:42:55.111467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:42:55.111551 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:42:55.112281 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:42:55.112284 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:42:55.112425 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 04:42:55.112575 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 04:42:55.113924 (+ 1349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":1459,"spinlock_wait_cycles":36736,"thread_start_us":59,"threads_started":1}
I20260809 04:42:55.115504 23343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:55.115161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:46987)
0809 04:42:55.115379 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:55.115425 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
W20260809 04:42:55.116256 22934 leader_election.cc:341] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109): Not found: Tablet not found: bd395a54b9354f8c815a0e5410e79766
W20260809 04:42:55.116460 22933 leader_election.cc:341] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693): Not found: Tablet not found: bd395a54b9354f8c815a0e5410e79766
I20260809 04:42:55.116559 22933 leader_election.cc:302] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 339f6af8d9a344918e391a084ddd0ab3; no voters: c2bc1810f85947818231f5d89ee41ed4, e6324db4e82b426fb0a26b7178377638
I20260809 04:42:55.116859 23339 raft_consensus.cc:2726] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:42:55.778155 23344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:55.777746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:58369)
0809 04:42:55.778004 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:55.778062 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":77,"threads_started":1}
I20260809 04:42:55.922755 23345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:55.922363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:53255)
0809 04:42:55.922612 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:55.922669 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":83,"threads_started":1}
I20260809 04:42:56.116667 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:56.116282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:51659)
0809 04:42:56.116512 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:56.116564 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
I20260809 04:42:56.779337 23347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:56.778939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:57477)
0809 04:42:56.779185 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:56.779241 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":78,"threads_started":1}
I20260809 04:42:56.923959 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:56.923485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:60469)
0809 04:42:56.923785 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:56.923842 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":73,"threads_started":1}
I20260809 04:42:57.059695 23349 raft_consensus.cc:482] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:42:57.059824 23349 raft_consensus.cc:504] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:57.060227 23349 leader_election.cc:288] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693), c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109)
W20260809 04:42:57.061195 22934 leader_election.cc:341] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109): Not found: Tablet not found: bd395a54b9354f8c815a0e5410e79766
W20260809 04:42:57.061287 22933 leader_election.cc:341] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693): Not found: Tablet not found: bd395a54b9354f8c815a0e5410e79766
I20260809 04:42:57.061331 22934 leader_election.cc:302] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 339f6af8d9a344918e391a084ddd0ab3; no voters: c2bc1810f85947818231f5d89ee41ed4, e6324db4e82b426fb0a26b7178377638
I20260809 04:42:57.061508 23349 raft_consensus.cc:2726] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:42:57.118006 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:57.117454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:39797)
0809 04:42:57.117806 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:57.117869 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":117,"threads_started":1}
I20260809 04:42:57.780561 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:57.780102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:59313)
0809 04:42:57.780396 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:57.780460 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":77,"threads_started":1}
I20260809 04:42:57.925136 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:57.924748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:54459)
0809 04:42:57.924988 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:57.925043 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
I20260809 04:42:58.119143 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:58.118673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:55299)
0809 04:42:58.118986 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:58.119048 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":89,"threads_started":1}
I20260809 04:42:58.781905 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:58.781477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:47825)
0809 04:42:58.781753 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:58.781811 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":74,"threads_started":1}
I20260809 04:42:58.926338 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:58.925930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:50555)
0809 04:42:58.926185 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:58.926244 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":79,"threads_started":1}
I20260809 04:42:59.120393 23356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:59.119974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:58649)
0809 04:42:59.120244 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:59.120301 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":103,"threads_started":1}
I20260809 04:42:59.271035 23357 raft_consensus.cc:482] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:42:59.271193 23357 raft_consensus.cc:504] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:42:59.271648 23357 leader_election.cc:288] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693), c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109)
W20260809 04:42:59.272547 22933 leader_election.cc:341] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693): Not found: Tablet not found: bd395a54b9354f8c815a0e5410e79766
W20260809 04:42:59.272710 22934 leader_election.cc:341] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109): Not found: Tablet not found: bd395a54b9354f8c815a0e5410e79766
I20260809 04:42:59.272863 22934 leader_election.cc:302] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 339f6af8d9a344918e391a084ddd0ab3; no voters: c2bc1810f85947818231f5d89ee41ed4, e6324db4e82b426fb0a26b7178377638
I20260809 04:42:59.273015 23357 raft_consensus.cc:2726] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:42:59.783245 23358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:59.782801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:59099)
0809 04:42:59.783078 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:59.783143 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":96,"threads_started":1}
I20260809 04:42:59.927702 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:42:59.927221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:53811)
0809 04:42:59.927508 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 04:42:59.927569 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":109,"threads_started":1}
I20260809 04:43:00.121641 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:00.121220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:57523)
0809 04:43:00.121489 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:00.121549 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":96,"threads_started":1}
I20260809 04:43:00.784458 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:00.784041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:54067)
0809 04:43:00.784293 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:00.784347 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
I20260809 04:43:00.929010 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:00.928482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:49439)
0809 04:43:00.928809 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:00.928898 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":97,"threads_started":1}
I20260809 04:43:01.122900 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:01.122444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:34279)
0809 04:43:01.122746 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:01.122805 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":83,"threads_started":1}
I20260809 04:43:01.785753 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:01.785333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:34461)
0809 04:43:01.785597 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:01.785654 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":82,"threads_started":1}
I20260809 04:43:01.930239 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:01.929757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:49037)
0809 04:43:01.930029 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:01.930089 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":77,"threads_started":1}
I20260809 04:43:01.930402 23367 raft_consensus.cc:482] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:01.930542 23367 raft_consensus.cc:504] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:43:01.930909 23367 leader_election.cc:288] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693), c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109)
W20260809 04:43:01.931659 22934 leader_election.cc:341] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109): Not found: Tablet not found: bd395a54b9354f8c815a0e5410e79766
W20260809 04:43:01.931763 22933 leader_election.cc:341] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693): Not found: Tablet not found: bd395a54b9354f8c815a0e5410e79766
I20260809 04:43:01.931870 22933 leader_election.cc:302] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 339f6af8d9a344918e391a084ddd0ab3; no voters: c2bc1810f85947818231f5d89ee41ed4, e6324db4e82b426fb0a26b7178377638
I20260809 04:43:01.932060 23367 raft_consensus.cc:2726] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:43:02.124130 23368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:02.123665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:44779)
0809 04:43:02.123979 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:02.124036 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":134,"threads_started":1}
I20260809 04:43:02.786900 23369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:02.786496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:35157)
0809 04:43:02.786751 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:02.786805 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":76,"threads_started":1}
I20260809 04:43:02.931375 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:02.930955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:35403)
0809 04:43:02.931212 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:02.931274 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260809 04:43:03.125269 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:03.124869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:60127)
0809 04:43:03.125103 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:03.125177 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":76,"threads_started":1}
I20260809 04:43:03.788226 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:03.787803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:36907)
0809 04:43:03.788075 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:03.788130 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":104,"threads_started":1}
I20260809 04:43:03.932684 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:03.932227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:58199)
0809 04:43:03.932523 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:03.932586 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":82,"threads_started":1}
I20260809 04:43:04.126555 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:04.126101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:45769)
0809 04:43:04.126384 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:04.126459 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":103,"threads_started":1}
I20260809 04:43:04.789669 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:04.789059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:46573)
0809 04:43:04.789511 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:04.789568 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":282,"threads_started":1}
I20260809 04:43:04.934155 23377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:04.933617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:53903)
0809 04:43:04.934005 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:04.934060 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":100,"threads_started":1}
I20260809 04:43:05.127833 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:05.127399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:45539)
0809 04:43:05.127662 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:05.127735 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":77,"threads_started":1}
I20260809 04:43:05.791040 23379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:05.790529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:56383)
0809 04:43:05.790888 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:05.790945 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":104,"threads_started":1}
I20260809 04:43:05.935359 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:05.934890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:46879)
0809 04:43:05.935169 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:05.935231 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":73,"threads_started":1}
I20260809 04:43:06.129282 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:06.128689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:55879)
0809 04:43:06.129071 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:06.129127 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":157,"threads_started":1}
I20260809 04:43:06.792223 23382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:06.791808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:47513)
0809 04:43:06.792064 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:06.792125 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":85,"threads_started":1}
I20260809 04:43:06.936609 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:06.936156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:51133)
0809 04:43:06.936440 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:06.936505 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":85,"threads_started":1}
I20260809 04:43:07.130581 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:07.130073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:60405)
0809 04:43:07.130331 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:07.130431 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":85,"threads_started":1}
I20260809 04:43:07.793543 23385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:07.793064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:41791)
0809 04:43:07.793377 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:07.793445 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":141,"threads_started":1}
I20260809 04:43:07.937951 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:07.937473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:38943)
0809 04:43:07.937790 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:07.937853 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":79,"threads_started":1}
I20260809 04:43:08.131873 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:08.131339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:59883)
0809 04:43:08.131624 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:08.131713 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":92,"threads_started":1}
I20260809 04:43:08.761947 23388 raft_consensus.cc:482] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:08.762086 23388 raft_consensus.cc:504] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6324db4e82b426fb0a26b7178377638" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 33693 } } peers { permanent_uuid: "339f6af8d9a344918e391a084ddd0ab3" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41333 } } peers { permanent_uuid: "c2bc1810f85947818231f5d89ee41ed4" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43109 } }
I20260809 04:43:08.762490 23388 leader_election.cc:288] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693), c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109)
W20260809 04:43:08.763208 22934 leader_election.cc:341] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2bc1810f85947818231f5d89ee41ed4 (127.5.222.194:43109): Not found: Tablet not found: bd395a54b9354f8c815a0e5410e79766
W20260809 04:43:08.763304 22933 leader_election.cc:341] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6324db4e82b426fb0a26b7178377638 (127.5.222.195:33693): Not found: Tablet not found: bd395a54b9354f8c815a0e5410e79766
I20260809 04:43:08.763425 22933 leader_election.cc:302] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 339f6af8d9a344918e391a084ddd0ab3; no voters: c2bc1810f85947818231f5d89ee41ed4, e6324db4e82b426fb0a26b7178377638
I20260809 04:43:08.763579 23388 raft_consensus.cc:2726] T bd395a54b9354f8c815a0e5410e79766 P 339f6af8d9a344918e391a084ddd0ab3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:43:08.794822 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:08.794444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.193:43901)
0809 04:43:08.794677 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:08.794730 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":76,"threads_started":1}
I20260809 04:43:08.939319 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:08.938886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.194:43303)
0809 04:43:08.939167 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:08.939223 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":88,"threads_started":1}
I20260809 04:43:09.118083 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 22879
I20260809 04:43:09.127807 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 23056
I20260809 04:43:09.133251 23391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:09.132794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:38293 (local address 127.5.222.195:40289)
0809 04:43:09.133059 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:09.133115 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:38293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":94,"threads_started":1}
I20260809 04:43:09.135979 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 23196
2026-08-09T04:43:09Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19302 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:43:09 dist-test-slave-vxj2 krb5kdc[23394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:43:09 dist-test-slave-vxj2 krb5kdc[23394](info): set up 2 sockets
Aug 09 04:43:09 dist-test-slave-vxj2 krb5kdc[23394](info): commencing operation
krb5kdc: starting...
W20260809 04:43:10.239480 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:43:10 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:43:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:43:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:43:10.364590 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:44275
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:44275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:10.486209 23408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:10.486479 23408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:10.486577 23408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:10.490912 23408 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 04:43:10.490980 23408 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 04:43:10.491000 23408 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 04:43:10.491017 23408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:43:10.491040 23408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:10.491113 23408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:10.491154 23408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:43:10.491191 23408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:43:10.496588 23408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:44275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:44275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:43:10.497699 23408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:10.498759 23408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:10.508311 23416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:10.508378 23414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:10.508440 23408 server_base.cc:1034] running on GCE node
W20260809 04:43:10.508409 23413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:10.509122 23408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:10.510278 23408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:10.511483 23408 hybrid_clock.cc:648] HybridClock initialized: now 1786250590511453 us; error 41 us; skew 500 ppm
Aug 09 04:43:10 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250590, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:10.514420 23408 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:43:10.515398 23408 webserver.cc:460] Webserver started at http://127.5.222.254:33447/ using document root <none> and password file <none>
I20260809 04:43:10.515975 23408 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:10.516068 23408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:10.516309 23408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:10.518463 23408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "14d3dd5b39a748359814f3da25173f6d"
format_stamp: "Formatted at 2026-08-09 04:43:10 on dist-test-slave-vxj2"
server_key: "d0ddd280b7fab73a19bef23f18fb310d"
server_key_iv: "167181c79ad6d027165913fde2c2cb8b"
server_key_version: "encryptionkey@0"
I20260809 04:43:10.519048 23408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "14d3dd5b39a748359814f3da25173f6d"
format_stamp: "Formatted at 2026-08-09 04:43:10 on dist-test-slave-vxj2"
server_key: "d0ddd280b7fab73a19bef23f18fb310d"
server_key_iv: "167181c79ad6d027165913fde2c2cb8b"
server_key_version: "encryptionkey@0"
I20260809 04:43:10.522899 23408 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:43:10.525347 23423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:10.526655 23408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:43:10.526837 23408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "14d3dd5b39a748359814f3da25173f6d"
format_stamp: "Formatted at 2026-08-09 04:43:10 on dist-test-slave-vxj2"
server_key: "d0ddd280b7fab73a19bef23f18fb310d"
server_key_iv: "167181c79ad6d027165913fde2c2cb8b"
server_key_version: "encryptionkey@0"
I20260809 04:43:10.526960 23408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:10.545836 23408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:10.549006 23408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:10.549281 23408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:10.557883 23408 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:44275
I20260809 04:43:10.558190 23480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:44275 every 8 connection(s)
I20260809 04:43:10.558959 23408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:43:10.561440 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 23408
I20260809 04:43:10.561589 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:43:10.562073 6011 external_mini_cluster.cc:1428] Setting key faf7f8aa9dd09d103394d81532d11b27
I20260809 04:43:10.562146 23481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:10.568765 23481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Bootstrap starting.
Aug 09 04:43:10 dist-test-slave-vxj2 krb5kdc[23394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:43:10.571668 23481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:10.572479 23481 log.cc:824] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:10.574570 23481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: No bootstrap required, opened a new log
I20260809 04:43:10.576778 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:10.563469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.5.222.254:44275)
0809 04:43:10.563892 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:10.563907 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:10.563945 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:43:10.564612 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:10.564628 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:10.564959 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:10.565369 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:10.566182 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:10.567194 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:10.568079 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:10.568544 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:10.568719 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:10.571524 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:10.571529 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:10.571542 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:10.571572 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:10.573659 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:10.574147 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:10.574152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:10.574156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:10.574284 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:10.574587 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:10.574590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:10.574592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:10.575045 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:10.575267 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:10.575508 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 04:43:10.575933 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":122,"threads_started":1}
I20260809 04:43:10.578212 23481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } }
I20260809 04:43:10.578459 23481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:10.578542 23481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d3dd5b39a748359814f3da25173f6d, State: Initialized, Role: FOLLOWER
I20260809 04:43:10.578948 23481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } }
I20260809 04:43:10.579102 23481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:43:10.579187 23481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:43:10.579457 23481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:10.580428 23481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } }
I20260809 04:43:10.580780 23481 leader_election.cc:302] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14d3dd5b39a748359814f3da25173f6d; no voters:
I20260809 04:43:10.581112 23481 leader_election.cc:288] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:43:10.581349 23486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:10.581626 23486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 1 LEADER]: Becoming Leader. State: Replica: 14d3dd5b39a748359814f3da25173f6d, State: Running, Role: LEADER
I20260809 04:43:10.581964 23486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } }
I20260809 04:43:10.582460 23481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:43:10.583702 23487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14d3dd5b39a748359814f3da25173f6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } } }
I20260809 04:43:10.583849 23487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:10.584625 23488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14d3dd5b39a748359814f3da25173f6d. Latest consensus state: current_term: 1 leader_uuid: "14d3dd5b39a748359814f3da25173f6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } } }
I20260809 04:43:10.584757 23488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:10.585254 23495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:43:10.588060 23495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:43:10.593634 23495 catalog_manager.cc:1347] Generated new cluster ID: 2b1a82ce1457403b91e6bfba1a63733a
I20260809 04:43:10.593784 23495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:43:10.605707 23495 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:43:10.607029 23495 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:43:10.616988 23495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Generated new TSK 0
I20260809 04:43:10.617861 23495 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:43:10.656888 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:44275
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:10.769088 23507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:10.769361 23507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:10.769403 23507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:10.773378 23507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:10.773437 23507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:10.773516 23507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:43:10.778609 23507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:44275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:43:10.779618 23507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:10.780619 23507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:10.790464 23517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:10.790422 23515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:10.790441 23513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:10.790489 23507 server_base.cc:1034] running on GCE node
I20260809 04:43:10.791272 23507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:10.791886 23507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:10.793044 23507 hybrid_clock.cc:648] HybridClock initialized: now 1786250590793031 us; error 29 us; skew 500 ppm
Aug 09 04:43:10 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250590, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:10.795840 23507 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:43:10.796658 23507 webserver.cc:460] Webserver started at http://127.5.222.193:44621/ using document root <none> and password file <none>
I20260809 04:43:10.797263 23507 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:10.797330 23507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:10.797573 23507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:10.799551 23507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "986519d0eca94fc9ab1d2952a8e53b86"
format_stamp: "Formatted at 2026-08-09 04:43:10 on dist-test-slave-vxj2"
server_key: "87b5ea09dc3f202ab2fad65c62945d56"
server_key_iv: "0c0f25f82bb10c511466d53a5781de7d"
server_key_version: "encryptionkey@0"
I20260809 04:43:10.800110 23507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "986519d0eca94fc9ab1d2952a8e53b86"
format_stamp: "Formatted at 2026-08-09 04:43:10 on dist-test-slave-vxj2"
server_key: "87b5ea09dc3f202ab2fad65c62945d56"
server_key_iv: "0c0f25f82bb10c511466d53a5781de7d"
server_key_version: "encryptionkey@0"
I20260809 04:43:10.804441 23507 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 04:43:10.806880 23524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:10.808082 23507 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:43:10.808216 23507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "986519d0eca94fc9ab1d2952a8e53b86"
format_stamp: "Formatted at 2026-08-09 04:43:10 on dist-test-slave-vxj2"
server_key: "87b5ea09dc3f202ab2fad65c62945d56"
server_key_iv: "0c0f25f82bb10c511466d53a5781de7d"
server_key_version: "encryptionkey@0"
I20260809 04:43:10.808327 23507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:10.838151 23507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:10.841398 23507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:10.841604 23507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:10.842212 23507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:10.843061 23507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:10.843134 23507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:10.843209 23507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:10.843253 23507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:10.852252 23507 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:38981
I20260809 04:43:10.852277 23642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:38981 every 8 connection(s)
I20260809 04:43:10.853119 23507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:43:10.853905 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 23507
I20260809 04:43:10.854048 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:43:10.854405 6011 external_mini_cluster.cc:1428] Setting key ad9fc023f6150a0098d0fc7648be777c
Aug 09 04:43:10 dist-test-slave-vxj2 krb5kdc[23394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250590, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:43:10.868602 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:10.855088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:49075 (local address 127.5.222.254:44275)
0809 04:43:10.855240 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:10.855246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:10.856175 (+ 929us) server_negotiation.cc:407] Connection header received
0809 04:43:10.857552 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:10.857557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:10.857620 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:10.857707 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:10.859344 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:10.859861 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:10.861789 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:10.862026 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:10.862109 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:10.865003 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:10.865008 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:10.865012 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:10.865047 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:10.866423 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:10.866966 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:10.866968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:10.866969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:10.867043 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:10.867373 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:10.867375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:10.867377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:10.867674 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:10.867795 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:10.868366 (+ 571us) server_negotiation.cc:299] Negotiation successful
0809 04:43:10.868478 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:43:10.869907 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:10.855490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:49075)
0809 04:43:10.856030 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:10.856073 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:10.857311 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:10.857924 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:10.857935 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:10.858475 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:10.859178 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:10.859202 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:10.860009 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:10.860016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:10.861657 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:10.861673 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:10.862196 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:10.862203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:10.862471 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:10.863107 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:10.863130 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:10.864819 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:10.866542 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:10.866547 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:10.866559 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:10.866807 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:10.867125 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:10.867128 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:10.867129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:10.867262 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:10.867786 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:10.867791 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:10.868100 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 04:43:10.868321 (+ 221us) client_negotiation.cc:240] Negotiation successful
0809 04:43:10.868838 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":150,"threads_started":1}
I20260809 04:43:10.871162 23643 heartbeater.cc:347] Connected to a master server at 127.5.222.254:44275
I20260809 04:43:10.871435 23643 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:10.872026 23643 heartbeater.cc:511] Master 127.5.222.254:44275 requested a full tablet report, sending...
I20260809 04:43:10.873723 23443 ts_manager.cc:194] Registered new tserver with Master: 986519d0eca94fc9ab1d2952a8e53b86 (127.5.222.193:38981)
I20260809 04:43:10.874866 23443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:49075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:43:10.884960 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:44275
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:10.997079 23648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:10.997417 23648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:10.997464 23648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:11.001418 23648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:11.001473 23648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:11.001533 23648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:43:11.006937 23648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:44275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:43:11.008033 23648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:11.009066 23648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:11.019064 23654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:11.019078 23656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:11.019093 23653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:11.019351 23648 server_base.cc:1034] running on GCE node
I20260809 04:43:11.019763 23648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:11.020347 23648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:11.021503 23648 hybrid_clock.cc:648] HybridClock initialized: now 1786250591021486 us; error 27 us; skew 500 ppm
Aug 09 04:43:11 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250591, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:11.027267 23648 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:43:11.028124 23648 webserver.cc:460] Webserver started at http://127.5.222.194:33161/ using document root <none> and password file <none>
I20260809 04:43:11.028677 23648 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:11.028738 23648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:11.028980 23648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:11.031179 23648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "755f2395f65b48a5a32c2c7e9d534e71"
format_stamp: "Formatted at 2026-08-09 04:43:11 on dist-test-slave-vxj2"
server_key: "b3f38d156b81de20f06f873855552a45"
server_key_iv: "3323938608c4d5a923a5c0c5f7d3f762"
server_key_version: "encryptionkey@0"
I20260809 04:43:11.031808 23648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "755f2395f65b48a5a32c2c7e9d534e71"
format_stamp: "Formatted at 2026-08-09 04:43:11 on dist-test-slave-vxj2"
server_key: "b3f38d156b81de20f06f873855552a45"
server_key_iv: "3323938608c4d5a923a5c0c5f7d3f762"
server_key_version: "encryptionkey@0"
I20260809 04:43:11.035439 23648 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 04:43:11.037767 23665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:11.038995 23648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:43:11.039113 23648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "755f2395f65b48a5a32c2c7e9d534e71"
format_stamp: "Formatted at 2026-08-09 04:43:11 on dist-test-slave-vxj2"
server_key: "b3f38d156b81de20f06f873855552a45"
server_key_iv: "3323938608c4d5a923a5c0c5f7d3f762"
server_key_version: "encryptionkey@0"
I20260809 04:43:11.039213 23648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:11.048345 23648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:11.051165 23648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:11.051352 23648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:11.051993 23648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:11.052776 23648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:11.052824 23648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:11.052860 23648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:11.052927 23648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:11.061658 23648 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:43097
I20260809 04:43:11.061683 23786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:43097 every 8 connection(s)
I20260809 04:43:11.062533 23648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
Aug 09 04:43:11 dist-test-slave-vxj2 krb5kdc[23394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250591, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:43:11.072254 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 23648
I20260809 04:43:11.072402 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:43:11.072702 6011 external_mini_cluster.cc:1428] Setting key 99d9a73f41abf40ada45ad127f7f006f
I20260809 04:43:11.076727 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.064361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:44683 (local address 127.5.222.254:44275)
0809 04:43:11.064505 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:11.064509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:11.065414 (+ 905us) server_negotiation.cc:407] Connection header received
0809 04:43:11.066784 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:11.066788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:11.066856 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:11.066959 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:11.068119 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.068652 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.069838 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.070069 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.070181 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.072883 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:11.072887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:11.072889 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:11.072914 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:11.074485 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:11.074997 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:11.075000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:11.075002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:11.075083 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:11.075428 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:11.075431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:11.075433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:11.075610 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:11.075717 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:11.076373 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 04:43:11.076514 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:43:11.077761 23789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.064732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:44683)
0809 04:43:11.065263 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:11.065306 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:11.066523 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:11.067136 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:11.067143 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:11.067500 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:11.067960 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.067973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.068784 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.068790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.069705 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.069715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.070222 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.070226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.070403 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.071052 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:11.071068 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:11.072700 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:11.074605 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:11.074611 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:11.074625 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:11.074880 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:11.075176 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:11.075179 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:11.075181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:11.075321 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:11.075717 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:11.075723 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:11.076060 (+ 337us) client_negotiation.cc:769] Sending connection context
0809 04:43:11.076294 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 04:43:11.076681 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":116,"threads_started":1}
I20260809 04:43:11.079008 23787 heartbeater.cc:347] Connected to a master server at 127.5.222.254:44275
I20260809 04:43:11.079283 23787 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:11.079780 23787 heartbeater.cc:511] Master 127.5.222.254:44275 requested a full tablet report, sending...
I20260809 04:43:11.080827 23443 ts_manager.cc:194] Registered new tserver with Master: 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097)
I20260809 04:43:11.081562 23443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:44683
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:43:11.100700 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:44275
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:11.216974 23792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:11.217312 23792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:11.217398 23792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:11.221452 23792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:11.221534 23792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:11.221619 23792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:43:11.227015 23792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:44275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:43:11.228111 23792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:11.229141 23792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:11.239359 23800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:11.239388 23798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:11.239574 23792 server_base.cc:1034] running on GCE node
W20260809 04:43:11.239442 23797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:11.240116 23792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:11.240770 23792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:11.241966 23792 hybrid_clock.cc:648] HybridClock initialized: now 1786250591241933 us; error 46 us; skew 500 ppm
Aug 09 04:43:11 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250591, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:11.244756 23792 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:43:11.245703 23792 webserver.cc:460] Webserver started at http://127.5.222.195:44117/ using document root <none> and password file <none>
I20260809 04:43:11.246279 23792 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:11.246377 23792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:11.246629 23792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:11.248649 23792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "8b4dc940ba00496ab450564bcd04df94"
format_stamp: "Formatted at 2026-08-09 04:43:11 on dist-test-slave-vxj2"
server_key: "74d3c6f890ac7c300c906eece25fe8e6"
server_key_iv: "121e7718c8fcd92e4ac970c9bfdc94fe"
server_key_version: "encryptionkey@0"
I20260809 04:43:11.249284 23792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "8b4dc940ba00496ab450564bcd04df94"
format_stamp: "Formatted at 2026-08-09 04:43:11 on dist-test-slave-vxj2"
server_key: "74d3c6f890ac7c300c906eece25fe8e6"
server_key_iv: "121e7718c8fcd92e4ac970c9bfdc94fe"
server_key_version: "encryptionkey@0"
I20260809 04:43:11.253065 23792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260809 04:43:11.255426 23808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:11.256598 23792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:43:11.256733 23792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "8b4dc940ba00496ab450564bcd04df94"
format_stamp: "Formatted at 2026-08-09 04:43:11 on dist-test-slave-vxj2"
server_key: "74d3c6f890ac7c300c906eece25fe8e6"
server_key_iv: "121e7718c8fcd92e4ac970c9bfdc94fe"
server_key_version: "encryptionkey@0"
I20260809 04:43:11.256882 23792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:11.267778 23792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:11.271039 23792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:11.271242 23792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:11.271838 23792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:11.272631 23792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:11.272703 23792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:11.272771 23792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:11.272826 23792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:11.281622 23792 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:40233
I20260809 04:43:11.281791 23927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:40233 every 8 connection(s)
I20260809 04:43:11.282531 23792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:43:11.287273 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 23792
I20260809 04:43:11.287428 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:43:11.287714 6011 external_mini_cluster.cc:1428] Setting key 5ef9ecd2ba86561a26ba44c6c875c2cc
Aug 09 04:43:11 dist-test-slave-vxj2 krb5kdc[23394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250591, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:43:11.296478 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.284388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:56779 (local address 127.5.222.254:44275)
0809 04:43:11.284535 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:11.284539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:11.285398 (+ 859us) server_negotiation.cc:407] Connection header received
0809 04:43:11.286666 (+ 1268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:11.286669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:11.286729 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:11.286829 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:11.288093 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.288808 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.290169 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.290388 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.290503 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.293067 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:11.293070 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:11.293072 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:11.293097 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:11.294401 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:11.294926 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:11.294933 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:11.294936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:11.295018 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:11.295368 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:11.295371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:11.295373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:11.295563 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:11.295670 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:11.296169 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 04:43:11.296295 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:43:11.297325 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.284780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:56779)
0809 04:43:11.285254 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:11.285296 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:11.286453 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:11.287004 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:11.287028 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:11.287368 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:11.287946 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.287961 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.288938 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.288941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.290042 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.290052 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.290541 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.290546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.290716 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.291339 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:11.291355 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:11.292885 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:11.294533 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:11.294539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:11.294553 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:11.294818 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:11.295113 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:11.295117 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:11.295119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:11.295260 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:11.295682 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:11.295690 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:11.295940 (+ 250us) client_negotiation.cc:769] Sending connection context
0809 04:43:11.296128 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 04:43:11.296446 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":112,"threads_started":1}
I20260809 04:43:11.298465 23928 heartbeater.cc:347] Connected to a master server at 127.5.222.254:44275
I20260809 04:43:11.298672 23928 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:11.299106 23928 heartbeater.cc:511] Master 127.5.222.254:44275 requested a full tablet report, sending...
I20260809 04:43:11.300067 23443 ts_manager.cc:194] Registered new tserver with Master: 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233)
I20260809 04:43:11.300808 23443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:56779
I20260809 04:43:11.301646 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:43:11.311388 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.303684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56132 (local address 127.5.222.254:44275)
0809 04:43:11.303796 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:11.303799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:11.304009 (+ 210us) server_negotiation.cc:407] Connection header received
0809 04:43:11.304101 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:11.304103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:11.304136 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:11.304197 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:11.304867 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.305500 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.306315 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.306535 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.306608 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.307883 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:11.307886 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:11.307888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:11.307909 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:11.309604 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:11.310036 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:11.310038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:11.310040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:11.310113 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:11.310506 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:11.310509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:11.310510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:11.310771 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:11.310886 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:11.311172 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 04:43:11.311273 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 04:43:11.314383 23443 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:43:11.317377 23443 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:43:11.332129 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.325703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:40233 (local address 127.0.0.1:46586)
0809 04:43:11.326911 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:11.326938 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:11.327038 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:11.327543 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:11.327547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:11.327564 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:11.327705 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.327716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.329128 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.329131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.330383 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.330393 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.331548 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.331551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.331673 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.331730 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 04:43:11.331839 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 04:43:11.331967 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1137,"mutex_wait_us":163,"thread_start_us":128,"threads_started":1}
I20260809 04:43:11.332149 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.326050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46586 (local address 127.5.222.195:40233)
0809 04:43:11.326965 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:11.326975 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:11.327001 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:43:11.327199 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:11.327204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:11.327413 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:11.327570 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:11.327824 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.328956 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.330562 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.331409 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.331534 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.331746 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:11.331887 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 04:43:11.331992 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":814,"thread_start_us":101,"threads_started":1}
I20260809 04:43:11.332376 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.325549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43097 (local address 127.0.0.1:36028)
0809 04:43:11.326240 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:11.326328 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:11.326668 (+ 340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:11.327340 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:11.327346 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:11.327368 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:11.327576 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.327585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.329128 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.329131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.330829 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.330836 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.331994 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.331997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.332080 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.332100 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:43:11.332170 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 04:43:11.332250 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":344,"thread_start_us":92,"threads_started":1}
I20260809 04:43:11.332756 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.325723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36028 (local address 127.5.222.194:43097)
0809 04:43:11.326676 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:11.326689 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:11.326716 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:43:11.326886 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:11.326893 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:11.327121 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:11.327310 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:11.327718 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.328983 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.330982 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.331668 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.331786 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.331933 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:11.332469 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 04:43:11.332579 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838,"thread_start_us":153,"threads_started":1}
I20260809 04:43:11.333204 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.326548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.5.222.193:38981)
0809 04:43:11.327048 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:11.327065 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:11.327261 (+ 196us) server_negotiation.cc:407] Connection header received
0809 04:43:11.327423 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:11.327428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:11.327609 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:11.327773 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:11.328146 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.328881 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.330967 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.331770 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.331909 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.332048 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:11.332943 (+ 895us) server_negotiation.cc:299] Negotiation successful
0809 04:43:11.333051 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":114,"threads_started":1}
I20260809 04:43:11.333490 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.326271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:38981 (local address 127.0.0.1:57338)
0809 04:43:11.327138 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:11.327153 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:11.327265 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:11.327929 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:11.327931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:11.327945 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:11.328020 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.328024 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.329064 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.329068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.330824 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.330836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.332114 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.332118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.332226 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.332250 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:43:11.332327 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 04:43:11.333314 (+ 987us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":187,"thread_start_us":94,"threads_started":1}
I20260809 04:43:11.335600 23859 tablet_service.cc:1467] Processing CreateTablet for tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71 (DEFAULT_TABLE table=test-table [id=7f6f58b0dd2f49b3b46d88dbbba34eef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:11.335712 23714 tablet_service.cc:1467] Processing CreateTablet for tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71 (DEFAULT_TABLE table=test-table [id=7f6f58b0dd2f49b3b46d88dbbba34eef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:11.335924 23575 tablet_service.cc:1467] Processing CreateTablet for tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71 (DEFAULT_TABLE table=test-table [id=7f6f58b0dd2f49b3b46d88dbbba34eef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:11.336706 23859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:11.336750 23575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:11.336760 23714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:11.343233 23946 tablet_bootstrap.cc:492] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94: Bootstrap starting.
I20260809 04:43:11.343847 23947 tablet_bootstrap.cc:492] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71: Bootstrap starting.
I20260809 04:43:11.345119 23948 tablet_bootstrap.cc:492] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Bootstrap starting.
I20260809 04:43:11.345744 23946 tablet_bootstrap.cc:654] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:11.346396 23947 tablet_bootstrap.cc:654] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:11.347146 23948 tablet_bootstrap.cc:654] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:11.347126 23947 log.cc:824] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:11.347126 23946 log.cc:824] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:11.348315 23948 log.cc:824] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:11.349129 23947 tablet_bootstrap.cc:492] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71: No bootstrap required, opened a new log
I20260809 04:43:11.349176 23946 tablet_bootstrap.cc:492] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94: No bootstrap required, opened a new log
I20260809 04:43:11.349352 23946 ts_tablet_manager.cc:1397] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:43:11.349357 23947 ts_tablet_manager.cc:1397] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260809 04:43:11.350286 23948 tablet_bootstrap.cc:492] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: No bootstrap required, opened a new log
I20260809 04:43:11.350461 23948 ts_tablet_manager.cc:1397] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 04:43:11.352262 23946 raft_consensus.cc:348] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:11.352511 23946 raft_consensus.cc:374] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:11.352567 23946 raft_consensus.cc:729] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b4dc940ba00496ab450564bcd04df94, State: Initialized, Role: FOLLOWER
I20260809 04:43:11.353016 23946 consensus_queue.cc:260] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:11.353230 23947 raft_consensus.cc:348] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:11.353691 23928 heartbeater.cc:503] Master 127.5.222.254:44275 was elected leader, sending a full tablet report...
I20260809 04:43:11.354003 23946 ts_tablet_manager.cc:1428] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:43:11.354149 23948 raft_consensus.cc:348] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:11.354380 23947 raft_consensus.cc:374] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:11.354410 23948 raft_consensus.cc:374] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:11.354460 23948 raft_consensus.cc:729] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 986519d0eca94fc9ab1d2952a8e53b86, State: Initialized, Role: FOLLOWER
I20260809 04:43:11.354451 23947 raft_consensus.cc:729] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755f2395f65b48a5a32c2c7e9d534e71, State: Initialized, Role: FOLLOWER
I20260809 04:43:11.354894 23947 consensus_queue.cc:260] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:11.354894 23948 consensus_queue.cc:260] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:11.355540 23643 heartbeater.cc:503] Master 127.5.222.254:44275 was elected leader, sending a full tablet report...
I20260809 04:43:11.355909 23947 ts_tablet_manager.cc:1428] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260809 04:43:11.356489 23787 heartbeater.cc:503] Master 127.5.222.254:44275 was elected leader, sending a full tablet report...
I20260809 04:43:11.357720 23948 ts_tablet_manager.cc:1428] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260809 04:43:11.533597 23929 tablet.cc:2369] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:43:11.563675 23788 tablet.cc:2369] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:43:11.604658 23644 tablet.cc:2369] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:43:11.636077 23954 raft_consensus.cc:482] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:11.636281 23954 raft_consensus.cc:504] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:11.637427 23954 leader_election.cc:288] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097), 986519d0eca94fc9ab1d2952a8e53b86 (127.5.222.193:38981)
I20260809 04:43:11.642030 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.637911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:44663 (local address 127.5.222.194:43097)
0809 04:43:11.638091 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:11.638096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:11.638114 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:43:11.638175 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:11.638178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:11.638229 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:11.638341 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:11.638648 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.639410 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.640854 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.641453 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.641570 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.641622 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:11.641807 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 04:43:11.641886 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 04:43:11.642057 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.637751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43097 (local address 127.5.222.195:44663)
0809 04:43:11.637899 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:11.637916 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:11.638024 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:11.638376 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:11.638379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:11.638399 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:11.638534 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.638542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.639532 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.639535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.640728 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.640739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.641568 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.641571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.641656 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.641676 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:43:11.641790 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 04:43:11.641867 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 04:43:11.642257 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.638232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:52205 (local address 127.5.222.193:38981)
0809 04:43:11.638380 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:11.638384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:11.638555 (+ 171us) server_negotiation.cc:407] Connection header received
0809 04:43:11.638653 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:11.638656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:11.638703 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:11.638785 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:11.639103 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.639716 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.641031 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.641629 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.641729 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.641797 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:11.642035 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 04:43:11.642129 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:43:11.642735 23957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.638076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:38981 (local address 127.5.222.195:52205)
0809 04:43:11.638453 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:11.638465 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:11.638564 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:11.638799 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:11.638801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:11.638815 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:11.638983 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.638992 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.639815 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.639819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.640924 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:11.640930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.641749 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.641754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:11.641889 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.641910 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:43:11.642449 (+ 539us) client_negotiation.cc:240] Negotiation successful
0809 04:43:11.642543 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":118,"threads_started":1}
I20260809 04:43:11.643319 23735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a3e0dc11de45a3bcd6e2ae5c7f9d71" candidate_uuid: "8b4dc940ba00496ab450564bcd04df94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "755f2395f65b48a5a32c2c7e9d534e71" is_pre_election: true
I20260809 04:43:11.643342 23596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a3e0dc11de45a3bcd6e2ae5c7f9d71" candidate_uuid: "8b4dc940ba00496ab450564bcd04df94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "986519d0eca94fc9ab1d2952a8e53b86" is_pre_election: true
I20260809 04:43:11.643631 23735 raft_consensus.cc:2454] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b4dc940ba00496ab450564bcd04df94 in term 0.
I20260809 04:43:11.643658 23596 raft_consensus.cc:2454] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b4dc940ba00496ab450564bcd04df94 in term 0.
I20260809 04:43:11.644119 23811 leader_election.cc:302] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b4dc940ba00496ab450564bcd04df94, 986519d0eca94fc9ab1d2952a8e53b86; no voters:
I20260809 04:43:11.644407 23954 raft_consensus.cc:2781] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:43:11.644485 23954 raft_consensus.cc:482] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:43:11.644541 23954 raft_consensus.cc:3037] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:11.645592 23954 raft_consensus.cc:504] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:11.645959 23954 leader_election.cc:288] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [CANDIDATE]: Term 1 election: Requested vote from peers 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097), 986519d0eca94fc9ab1d2952a8e53b86 (127.5.222.193:38981)
I20260809 04:43:11.646402 23596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a3e0dc11de45a3bcd6e2ae5c7f9d71" candidate_uuid: "8b4dc940ba00496ab450564bcd04df94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "986519d0eca94fc9ab1d2952a8e53b86"
I20260809 04:43:11.646402 23735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a3e0dc11de45a3bcd6e2ae5c7f9d71" candidate_uuid: "8b4dc940ba00496ab450564bcd04df94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "755f2395f65b48a5a32c2c7e9d534e71"
I20260809 04:43:11.646536 23735 raft_consensus.cc:3037] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:11.646536 23596 raft_consensus.cc:3037] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:11.647789 23735 raft_consensus.cc:2454] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b4dc940ba00496ab450564bcd04df94 in term 1.
I20260809 04:43:11.647789 23596 raft_consensus.cc:2454] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b4dc940ba00496ab450564bcd04df94 in term 1.
I20260809 04:43:11.648080 23811 leader_election.cc:302] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b4dc940ba00496ab450564bcd04df94, 986519d0eca94fc9ab1d2952a8e53b86; no voters:
I20260809 04:43:11.648236 23954 raft_consensus.cc:2781] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:11.648460 23954 raft_consensus.cc:686] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 1 LEADER]: Becoming Leader. State: Replica: 8b4dc940ba00496ab450564bcd04df94, State: Running, Role: LEADER
I20260809 04:43:11.648805 23954 consensus_queue.cc:237] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:11.652004 23442 catalog_manager.cc:5495] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195). New cstate: current_term: 1 leader_uuid: "8b4dc940ba00496ab450564bcd04df94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } health_report { overall_health: UNKNOWN } } }
I20260809 04:43:11.698067 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.694194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46602 (local address 127.5.222.195:40233)
0809 04:43:11.694357 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:11.694361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:11.694373 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:43:11.694445 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:11.694447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:11.694499 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:11.694593 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:11.694949 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.695458 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.696533 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:11.696756 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:11.696849 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:11.697209 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:11.697644 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:11.697741 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:11.697868 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 04:43:11.697936 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 04:43:11.704082 23596 raft_consensus.cc:1261] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Refusing update from remote peer 8b4dc940ba00496ab450564bcd04df94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:11.704118 23735 raft_consensus.cc:1261] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71 [term 1 FOLLOWER]: Refusing update from remote peer 8b4dc940ba00496ab450564bcd04df94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:11.704830 23958 consensus_queue.cc:1035] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:11.705046 23954 consensus_queue.cc:1035] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:11.715051 23963 mvcc.cc:205] Tried to move back new op lower bound from 7316482423612354560 to 7316482423396257792. Current Snapshot: MvccSnapshot[applied={T|T < 7316482423612354560 or (T in {7316482423612354560})}]
I20260809 04:43:11.716203 23962 mvcc.cc:205] Tried to move back new op lower bound from 7316482423612354560 to 7316482423396257792. Current Snapshot: MvccSnapshot[applied={T|T < 7316482423612354560}]
I20260809 04:43:11.721217 23443 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:11.721393 23443 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:11.723882 23443 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Sending DeleteTablet for 3 replicas of tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71
I20260809 04:43:11.724735 23714 tablet_service.cc:1514] Processing DeleteTablet for tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:11 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:36028
I20260809 04:43:11.724813 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 23507
I20260809 04:43:11.724876 23859 tablet_service.cc:1514] Processing DeleteTablet for tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:11 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:46586
I20260809 04:43:11.727990 23970 tablet_replica.cc:330] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71: stopping tablet replica
I20260809 04:43:11.727990 23971 tablet_replica.cc:330] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94: stopping tablet replica
I20260809 04:43:11.728319 23971 raft_consensus.cc:2229] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:43:11.728319 23970 raft_consensus.cc:2229] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:11.728646 23970 raft_consensus.cc:2258] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:11.728664 23971 raft_consensus.cc:2258] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:11.730100 23971 ts_tablet_manager.cc:1905] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:11.730826 23970 ts_tablet_manager.cc:1905] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:11.733028 23971 ts_tablet_manager.cc:1918] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:43:11.733130 23971 log.cc:1196] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/07a3e0dc11de45a3bcd6e2ae5c7f9d71
I20260809 04:43:11.733558 23971 ts_tablet_manager.cc:1939] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 8b4dc940ba00496ab450564bcd04df94: Deleting consensus metadata
I20260809 04:43:11.733629 23970 ts_tablet_manager.cc:1918] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:43:11.733726 23970 log.cc:1196] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/07a3e0dc11de45a3bcd6e2ae5c7f9d71
I20260809 04:43:11.734033 23970 ts_tablet_manager.cc:1939] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 755f2395f65b48a5a32c2c7e9d534e71: Deleting consensus metadata
I20260809 04:43:11.734899 23427 catalog_manager.cc:4841] TS 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233): tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71 (table test-table [id=7f6f58b0dd2f49b3b46d88dbbba34eef]) successfully deleted
I20260809 04:43:11.735138 23426 catalog_manager.cc:4841] TS 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097): tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71 (table test-table [id=7f6f58b0dd2f49b3b46d88dbbba34eef]) successfully deleted
W20260809 04:43:11.743922 23426 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:38981 (error 108)
I20260809 04:43:11.744196 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 23648
I20260809 04:43:11.746532 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:11.746187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:38981 (local address 127.0.0.1:57344)
0809 04:43:11.746330 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:11.746407 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 04:43:11.747776 23426 catalog_manager.cc:4584] TS 986519d0eca94fc9ab1d2952a8e53b86 (127.5.222.193:38981): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71: Network error: Client connection negotiation failed: client connection to 127.5.222.193:38981: connect: Connection refused (error 111)
I20260809 04:43:11.752673 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 23792
I20260809 04:43:11.760161 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 23408
I20260809 04:43:31.769228 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:44275
--webserver_interface=127.5.222.254
--webserver_port=33447
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:44275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:31.883653 23973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:31.883921 23973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:31.883961 23973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:31.888057 23973 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 04:43:31.888119 23973 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 04:43:31.888136 23973 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 04:43:31.888154 23973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:43:31.888175 23973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:31.888195 23973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:31.888212 23973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:43:31.888243 23973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:43:31.893698 23973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:44275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:44275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=33447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:43:31.894685 23973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:31.895735 23973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:43:31.909775 23973 server_base.cc:1034] running on GCE node
W20260809 04:43:31.909762 23980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:31.909718 23982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:31.909794 23979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:31.910763 23973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:31.911967 23973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:31.913137 23973 hybrid_clock.cc:648] HybridClock initialized: now 1786250611913118 us; error 28 us; skew 500 ppm
Aug 09 04:43:31 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250611, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:31.916245 23973 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:43:31.917303 23973 webserver.cc:460] Webserver started at http://127.5.222.254:33447/ using document root <none> and password file <none>
I20260809 04:43:31.917908 23973 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:31.917994 23973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:31.922015 23973 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:43:31.924165 23989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:31.925513 23973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:43:31.925663 23973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "14d3dd5b39a748359814f3da25173f6d"
format_stamp: "Formatted at 2026-08-09 04:43:10 on dist-test-slave-vxj2"
server_key: "d0ddd280b7fab73a19bef23f18fb310d"
server_key_iv: "167181c79ad6d027165913fde2c2cb8b"
server_key_version: "encryptionkey@0"
I20260809 04:43:31.926348 23973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:31.942013 23973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:31.945508 23973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:31.945724 23973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:31.953393 23973 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:44275
I20260809 04:43:31.953702 24045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:44275 every 8 connection(s)
I20260809 04:43:31.954629 23973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:43:31.956521 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 23973
I20260809 04:43:31.956974 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:38981
--local_ip_for_outbound_sockets=127.5.222.193
--tserver_master_addrs=127.5.222.254:44275
--webserver_port=44621
--webserver_interface=127.5.222.193
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:43:31.959369 24046 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:43:31.961333 24046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Bootstrap starting.
I20260809 04:43:31.986352 24046 log.cc:824] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:31.997776 24046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:43:31.998275 24046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Bootstrap complete.
I20260809 04:43:32.004153 24046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } }
I20260809 04:43:32.004683 24046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d3dd5b39a748359814f3da25173f6d, State: Initialized, Role: FOLLOWER
I20260809 04:43:32.005281 24046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } }
I20260809 04:43:32.005438 24046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:43:32.005519 24046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:43:32.005621 24046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:43:32.008296 24046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } }
I20260809 04:43:32.008695 24046 leader_election.cc:302] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14d3dd5b39a748359814f3da25173f6d; no voters:
I20260809 04:43:32.009114 24046 leader_election.cc:288] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:43:32.009272 24050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:43:32.009550 24050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 2 LEADER]: Becoming Leader. State: Replica: 14d3dd5b39a748359814f3da25173f6d, State: Running, Role: LEADER
I20260809 04:43:32.009871 24050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } }
I20260809 04:43:32.010478 24046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:43:32.012074 24051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "14d3dd5b39a748359814f3da25173f6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } } }
I20260809 04:43:32.012104 24052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14d3dd5b39a748359814f3da25173f6d. Latest consensus state: current_term: 2 leader_uuid: "14d3dd5b39a748359814f3da25173f6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } } }
I20260809 04:43:32.012207 24051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:32.012210 24052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:32.012600 24059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:43:32.017743 24059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:43:32.018633 24059 catalog_manager.cc:1259] Loaded cluster ID: 2b1a82ce1457403b91e6bfba1a63733a
I20260809 04:43:32.018711 24059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:43:32.020548 24059 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:43:32.021198 24059 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Loaded TSK: 0
I20260809 04:43:32.035795 24059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Generated new TSK 1
I20260809 04:43:32.036082 24059 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:43:32.100886 24048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:32.101213 24048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:32.101316 24048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:32.105835 24048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:32.105893 24048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:32.105991 24048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:43:32.111511 24048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:38981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=44621
--tserver_master_addrs=127.5.222.254:44275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:43:32.112605 24048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:32.113795 24048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:32.123589 24074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:32.123589 24073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:32.123634 24077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:32.123854 24048 server_base.cc:1034] running on GCE node
I20260809 04:43:32.124389 24048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:32.125031 24048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:32.126356 24048 hybrid_clock.cc:648] HybridClock initialized: now 1786250612126329 us; error 45 us; skew 500 ppm
Aug 09 04:43:32 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250612, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:32.129122 24048 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:43:32.130110 24048 webserver.cc:460] Webserver started at http://127.5.222.193:44621/ using document root <none> and password file <none>
I20260809 04:43:32.130723 24048 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:32.130800 24048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:32.134765 24048 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:43:32.137079 24084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:32.138476 24048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:43:32.138628 24048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "986519d0eca94fc9ab1d2952a8e53b86"
format_stamp: "Formatted at 2026-08-09 04:43:10 on dist-test-slave-vxj2"
server_key: "87b5ea09dc3f202ab2fad65c62945d56"
server_key_iv: "0c0f25f82bb10c511466d53a5781de7d"
server_key_version: "encryptionkey@0"
I20260809 04:43:32.139178 24048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:32.154206 24048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:32.157642 24048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:32.157862 24048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:32.158481 24048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:32.159688 24091 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 04:43:32.163902 24048 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 04:43:32.163975 24048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260809 04:43:32.164026 24048 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 04:43:32.165917 24048 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 04:43:32.165980 24048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260809 04:43:32.166245 24091 tablet_bootstrap.cc:492] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Bootstrap starting.
I20260809 04:43:32.177714 24048 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:38981
I20260809 04:43:32.177737 24204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:38981 every 8 connection(s)
I20260809 04:43:32.179059 24048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:43:32.186365 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 24048
I20260809 04:43:32.186848 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:43097
--local_ip_for_outbound_sockets=127.5.222.194
--tserver_master_addrs=127.5.222.254:44275
--webserver_port=33161
--webserver_interface=127.5.222.194
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:43:32.187729 24091 log.cc:824] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:43:32 dist-test-slave-vxj2 krb5kdc[23394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250612, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:43:32.196952 24091 tablet_bootstrap.cc:492] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:43:32.197311 24208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.181237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:57191)
0809 04:43:32.181830 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:32.181874 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:32.182711 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:32.183804 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:32.183812 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:32.184310 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:32.184761 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.184774 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.186212 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.186216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.187508 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.187520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.188616 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.188619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.188832 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.189535 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:32.189559 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:32.191465 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:32.193996 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:32.194010 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:32.194022 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:32.194291 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:32.194673 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:32.194676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:32.194678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:32.194853 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:32.195466 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:32.195472 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:32.195712 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 04:43:32.195911 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 04:43:32.196249 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":167,"threads_started":1}
I20260809 04:43:32.197484 24091 tablet_bootstrap.cc:492] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Bootstrap complete.
I20260809 04:43:32.197858 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.181262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:57191 (local address 127.5.222.254:44275)
0809 04:43:32.181639 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:32.181673 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:32.181999 (+ 326us) server_negotiation.cc:407] Connection header received
0809 04:43:32.183130 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:32.183140 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:32.183422 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:32.183760 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:32.184947 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.186009 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.187649 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.188462 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.188653 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.191672 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:32.191677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:32.191688 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:32.191717 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:32.193853 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:32.194421 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:32.194427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:32.194432 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:32.194567 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:32.194969 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:32.194973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:32.194975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:32.195336 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:32.195494 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:32.196041 (+ 547us) server_negotiation.cc:299] Negotiation successful
0809 04:43:32.196324 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":129,"threads_started":1}
I20260809 04:43:32.199267 24091 ts_tablet_manager.cc:1397] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Time spent bootstrapping tablet: real 0.033s user 0.019s sys 0.012s
I20260809 04:43:32.199949 24205 heartbeater.cc:347] Connected to a master server at 127.5.222.254:44275
I20260809 04:43:32.200196 24205 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:32.200685 24205 heartbeater.cc:511] Master 127.5.222.254:44275 requested a full tablet report, sending...
I20260809 04:43:32.202728 24091 raft_consensus.cc:348] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:32.203253 24091 raft_consensus.cc:729] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 986519d0eca94fc9ab1d2952a8e53b86, State: Initialized, Role: FOLLOWER
I20260809 04:43:32.203238 24008 ts_manager.cc:194] Registered new tserver with Master: 986519d0eca94fc9ab1d2952a8e53b86 (127.5.222.193:38981)
I20260809 04:43:32.203729 24091 consensus_queue.cc:260] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:32.204746 24091 ts_tablet_manager.cc:1428] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
I20260809 04:43:32.206967 24008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:57191
I20260809 04:43:32.209649 24205 heartbeater.cc:503] Master 127.5.222.254:44275 was elected leader, sending a full tablet report...
Aug 09 04:43:32 dist-test-slave-vxj2 krb5kdc[23394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250611, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.193@KRBTEST.COM
I20260809 04:43:32.217990 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.206306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:38981 (local address 127.0.0.1:59882)
0809 04:43:32.207055 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:32.207108 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:32.207254 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:32.207771 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:32.207776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:32.208281 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:32.208429 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.208438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.209857 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.209860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.210876 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.210890 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.211689 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.211693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.211815 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.212403 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:32.212418 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:32.214079 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:32.216086 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:32.216098 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:32.216103 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:32.216353 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:32.216686 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:32.216689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:32.216691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:32.216786 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:32.217311 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:32.217316 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:32.217447 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 04:43:32.217588 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 04:43:32.217803 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":105,"threads_started":1}
I20260809 04:43:32.217998 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.206383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59882 (local address 127.5.222.193:38981)
0809 04:43:32.206774 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:32.206786 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:32.207211 (+ 425us) server_negotiation.cc:407] Connection header received
0809 04:43:32.207407 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:32.207412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:32.207606 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:32.208088 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:32.208567 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.209658 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.211023 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.211566 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.211663 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.214235 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:32.214246 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:32.214258 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:32.214290 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:32.215963 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:32.216501 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:32.216505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:32.216506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:32.216587 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:32.216881 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:32.216885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:32.216887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:32.217126 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:32.217337 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:32.217684 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 04:43:32.217854 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":304,"thread_start_us":126,"threads_started":1}
I20260809 04:43:32.219174 24137 tablet_service.cc:1514] Processing DeleteTablet for tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:11 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:59882
I20260809 04:43:32.220032 24214 tablet_replica.cc:330] stopping tablet replica
I20260809 04:43:32.220345 24214 raft_consensus.cc:2229] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:32.220624 24214 raft_consensus.cc:2258] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:32.221669 24214 ts_tablet_manager.cc:1905] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:32.225452 24214 ts_tablet_manager.cc:1918] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:43:32.225536 24214 log.cc:1196] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal/wals/07a3e0dc11de45a3bcd6e2ae5c7f9d71
I20260809 04:43:32.225838 24214 ts_tablet_manager.cc:1939] T 07a3e0dc11de45a3bcd6e2ae5c7f9d71 P 986519d0eca94fc9ab1d2952a8e53b86: Deleting consensus metadata
I20260809 04:43:32.227497 23993 catalog_manager.cc:4841] TS 986519d0eca94fc9ab1d2952a8e53b86 (127.5.222.193:38981): tablet 07a3e0dc11de45a3bcd6e2ae5c7f9d71 (table test-table [id=7f6f58b0dd2f49b3b46d88dbbba34eef]) successfully deleted
W20260809 04:43:32.307628 24210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:32.307950 24210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:32.308039 24210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:32.312273 24210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:32.312350 24210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:32.312445 24210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:43:32.317735 24210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:43097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=33161
--tserver_master_addrs=127.5.222.254:44275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:43:32.318799 24210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:32.319835 24210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:32.329494 24223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:32.329483 24220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:32.329483 24219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:32.329548 24210 server_base.cc:1034] running on GCE node
I20260809 04:43:32.330143 24210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:32.330783 24210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:32.331974 24210 hybrid_clock.cc:648] HybridClock initialized: now 1786250612331958 us; error 34 us; skew 500 ppm
Aug 09 04:43:32 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250612, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:32.334872 24210 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:43:32.335789 24210 webserver.cc:460] Webserver started at http://127.5.222.194:33161/ using document root <none> and password file <none>
I20260809 04:43:32.336357 24210 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:32.336464 24210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:32.340265 24210 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:43:32.342398 24232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:32.344087 24210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:43:32.344224 24210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "755f2395f65b48a5a32c2c7e9d534e71"
format_stamp: "Formatted at 2026-08-09 04:43:11 on dist-test-slave-vxj2"
server_key: "b3f38d156b81de20f06f873855552a45"
server_key_iv: "3323938608c4d5a923a5c0c5f7d3f762"
server_key_version: "encryptionkey@0"
I20260809 04:43:32.344822 24210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:32.358482 24210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:32.362154 24210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:32.362370 24210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:32.362989 24210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:32.363843 24210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:32.363916 24210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:32.364012 24210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:32.364064 24210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:32.373373 24210 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:43097
I20260809 04:43:32.373411 24351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:43097 every 8 connection(s)
I20260809 04:43:32.374379 24210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:43:32.375964 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 24210
I20260809 04:43:32.376495 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:40233
--local_ip_for_outbound_sockets=127.5.222.195
--tserver_master_addrs=127.5.222.254:44275
--webserver_port=44117
--webserver_interface=127.5.222.195
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:43:32 dist-test-slave-vxj2 krb5kdc[23394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250612, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:43:32.389804 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.377418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:37473 (local address 127.5.222.254:44275)
0809 04:43:32.377559 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:32.377567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:32.377627 (+ 60us) server_negotiation.cc:407] Connection header received
0809 04:43:32.379116 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:32.379120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:32.379190 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:32.379311 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:32.380682 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.381350 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.382688 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.382958 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.383085 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.385805 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:32.385814 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:32.385816 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:32.385845 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:32.387452 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:32.388016 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:32.388020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:32.388022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:32.388110 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:32.388475 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:32.388480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:32.388482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:32.388666 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:32.388932 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:32.389487 (+ 555us) server_negotiation.cc:299] Negotiation successful
0809 04:43:32.389626 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:43:32.390594 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.376821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:37473)
0809 04:43:32.377460 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:32.377524 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:32.378864 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:32.379502 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:32.379509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:32.379911 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:32.380511 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.380525 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.381552 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.381556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.382520 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.382530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.383125 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.383129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.383339 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.384000 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:32.384018 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:32.385612 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:32.387592 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:32.387598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:32.387613 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:32.387899 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:32.388213 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:32.388218 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:32.388220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:32.388371 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:32.388918 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:32.388925 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:32.389229 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 04:43:32.389441 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 04:43:32.389749 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":151,"threads_started":1}
I20260809 04:43:32.391831 24352 heartbeater.cc:347] Connected to a master server at 127.5.222.254:44275
I20260809 04:43:32.392086 24352 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:32.392560 24352 heartbeater.cc:511] Master 127.5.222.254:44275 requested a full tablet report, sending...
I20260809 04:43:32.393661 24008 ts_manager.cc:194] Registered new tserver with Master: 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097)
I20260809 04:43:32.394220 24008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:37473
W20260809 04:43:32.492729 24356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:32.493024 24356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:32.493101 24356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:32.497359 24356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:32.497442 24356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:32.497532 24356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:43:32.502794 24356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:40233
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=44117
--tserver_master_addrs=127.5.222.254:44275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:43:32.503804 24356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:32.504843 24356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:32.514492 24362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:32.514559 24361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:32.514590 24356 server_base.cc:1034] running on GCE node
W20260809 04:43:32.514489 24364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:32.515086 24356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:32.515671 24356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:32.516826 24356 hybrid_clock.cc:648] HybridClock initialized: now 1786250612516808 us; error 30 us; skew 500 ppm
Aug 09 04:43:32 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250612, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:32.519651 24356 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:43:32.520565 24356 webserver.cc:460] Webserver started at http://127.5.222.195:44117/ using document root <none> and password file <none>
I20260809 04:43:32.521123 24356 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:32.521287 24356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:32.525048 24356 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:43:32.527058 24372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:32.528288 24356 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:43:32.528421 24356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "8b4dc940ba00496ab450564bcd04df94"
format_stamp: "Formatted at 2026-08-09 04:43:11 on dist-test-slave-vxj2"
server_key: "74d3c6f890ac7c300c906eece25fe8e6"
server_key_iv: "121e7718c8fcd92e4ac970c9bfdc94fe"
server_key_version: "encryptionkey@0"
I20260809 04:43:32.528952 24356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:32.558219 24356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:32.561467 24356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:32.561676 24356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:32.562288 24356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:32.563136 24356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:32.563210 24356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:32.563283 24356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:32.563338 24356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:32.572991 24356 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:40233
I20260809 04:43:32.573012 24490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:40233 every 8 connection(s)
I20260809 04:43:32.574036 24356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:43:32.574817 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 24356
I20260809 04:43:32.583453 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.575576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48706 (local address 127.5.222.254:44275)
0809 04:43:32.575717 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:32.575723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:32.575952 (+ 229us) server_negotiation.cc:407] Connection header received
0809 04:43:32.576131 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:32.576137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:32.576196 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:32.576279 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:32.577057 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.577709 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.578596 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.578829 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.578941 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.580343 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:32.580346 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:32.580349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:32.580381 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:32.581808 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:32.582232 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:32.582239 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:32.582242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:32.582335 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:32.582621 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:32.582625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:32.582627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:32.582796 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:32.583018 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:32.583164 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 04:43:32.583275 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 04:43:32 dist-test-slave-vxj2 krb5kdc[23394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250612, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:43:32.587819 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.576303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:35471 (local address 127.5.222.254:44275)
0809 04:43:32.577284 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:32.577288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:32.577301 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:43:32.578287 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:32.578290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:32.578337 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:32.578426 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:32.579559 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.580067 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.581581 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.581816 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.581927 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.584462 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:32.584465 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:32.584470 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:32.584496 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:32.586050 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:32.586529 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:32.586531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:32.586532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:32.586606 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:32.586952 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:32.586954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:32.586955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:32.587109 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:32.587185 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:32.587610 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 04:43:32.587704 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":913,"thread_start_us":80,"threads_started":1}
I20260809 04:43:32.588660 24495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.576313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:35471)
0809 04:43:32.576833 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:32.576884 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:32.578073 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:32.578604 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:32.578611 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:32.578942 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:32.579391 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.579403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.580200 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.580204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.581447 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.581457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.581967 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.581971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.582149 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.582689 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:32.582705 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:32.584303 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:32.586154 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:32.586158 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:32.586169 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:32.586421 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:32.586683 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:32.586685 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:32.586686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:32.586809 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:32.587194 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:32.587198 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:32.587412 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 04:43:32.587561 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 04:43:32.587794 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":163,"threads_started":1}
I20260809 04:43:32.589848 24491 heartbeater.cc:347] Connected to a master server at 127.5.222.254:44275
I20260809 04:43:32.590085 24491 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:32.590552 24491 heartbeater.cc:511] Master 127.5.222.254:44275 requested a full tablet report, sending...
I20260809 04:43:32.591558 24008 ts_manager.cc:194] Registered new tserver with Master: 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233)
I20260809 04:43:32.592135 24008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:35471
I20260809 04:43:32.597211 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:43:32.601039 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.597831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48718 (local address 127.5.222.254:44275)
0809 04:43:32.597961 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:32.597964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:32.598147 (+ 183us) server_negotiation.cc:407] Connection header received
0809 04:43:32.598283 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:32.598285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:32.598376 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:32.598462 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:32.598681 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.599182 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.599994 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.600205 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.600279 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.600604 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:32.600729 (+ 125us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 04:43:32.600942 (+ 213us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48718: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 04:43:32.601125 24494 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48718: token expired
W20260809 04:43:32.601567 23934 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.222.254:44275)
I20260809 04:43:32.608629 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.602036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48720 (local address 127.5.222.254:44275)
0809 04:43:32.602154 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:32.602157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:32.602169 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:43:32.602221 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:32.602223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:32.602256 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:32.602321 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:32.603111 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.603569 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.604332 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.604532 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.604605 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.605968 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:32.605971 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:32.605972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:32.605991 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:32.607282 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:32.607651 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:32.607654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:32.607655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:32.607727 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:32.607962 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:32.607964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:32.607965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:32.608117 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:32.608217 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:32.608415 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 04:43:32.608518 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:43:32.611057 24008 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:43:32.614600 24008 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:43:32.622879 24137 tablet_service.cc:1467] Processing CreateTablet for tablet d7130033ea9c4e58997d85f1e18d0929 (DEFAULT_TABLE table=test-table [id=84fb6049d5fc431098f7d41b2498ac61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:32.623602 24137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7130033ea9c4e58997d85f1e18d0929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:32.626996 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.621704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43097 (local address 127.0.0.1:46996)
0809 04:43:32.621876 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:32.621891 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:32.621992 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:32.622941 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:32.622944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:32.622962 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:32.623081 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.623089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.624330 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.624344 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.625308 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.625318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.626531 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.626534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.626640 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.626697 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 04:43:32.626770 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 04:43:32.626865 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 04:43:32.627226 24091 tablet_bootstrap.cc:492] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86: Bootstrap starting.
I20260809 04:43:32.627534 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.621920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:40233 (local address 127.0.0.1:47590)
0809 04:43:32.622315 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:32.622329 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:32.622414 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:32.623354 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:32.623357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:32.623375 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:32.623493 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.623499 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.625069 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.625072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.625949 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.625961 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.627104 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.627108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.627213 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.627235 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:43:32.627319 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:43:32.627401 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":89,"threads_started":1}
I20260809 04:43:32.627583 24503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.622316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47590 (local address 127.5.222.195:40233)
0809 04:43:32.622689 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:32.622715 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:32.622738 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:43:32.622813 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:32.622818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:32.623010 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:32.623644 (+ 634us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:32.623819 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.624943 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.626097 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.626981 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.627100 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.627243 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:32.627352 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 04:43:32.627444 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":100,"threads_started":1}
I20260809 04:43:32.628154 24091 tablet_bootstrap.cc:654] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:32.629807 24091 tablet_bootstrap.cc:492] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86: No bootstrap required, opened a new log
I20260809 04:43:32.629899 24091 ts_tablet_manager.cc:1397] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 04:43:32.630386 24091 raft_consensus.cc:348] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:32.630496 24091 raft_consensus.cc:374] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:32.630525 24091 raft_consensus.cc:729] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 986519d0eca94fc9ab1d2952a8e53b86, State: Initialized, Role: FOLLOWER
I20260809 04:43:32.630676 24091 consensus_queue.cc:260] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:32.630961 24091 ts_tablet_manager.cc:1428] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 04:43:32.631451 24501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.621870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46996 (local address 127.5.222.194:43097)
0809 04:43:32.622456 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:32.622475 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:32.622502 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:43:32.622587 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:32.622594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:32.622777 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:32.622961 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:32.623191 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.624211 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.625454 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.626343 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.629266 (+ 2923us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.631141 (+ 1875us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:32.631187 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 04:43:32.631262 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":153,"threads_started":1}
I20260809 04:43:32.630987 24424 tablet_service.cc:1467] Processing CreateTablet for tablet d7130033ea9c4e58997d85f1e18d0929 (DEFAULT_TABLE table=test-table [id=84fb6049d5fc431098f7d41b2498ac61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:32.632077 24424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7130033ea9c4e58997d85f1e18d0929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:32.634481 24284 tablet_service.cc:1467] Processing CreateTablet for tablet d7130033ea9c4e58997d85f1e18d0929 (DEFAULT_TABLE table=test-table [id=84fb6049d5fc431098f7d41b2498ac61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:32.635368 24284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7130033ea9c4e58997d85f1e18d0929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:32.640149 24505 tablet_bootstrap.cc:492] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94: Bootstrap starting.
I20260809 04:43:32.642268 24506 tablet_bootstrap.cc:492] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71: Bootstrap starting.
I20260809 04:43:32.642987 24505 tablet_bootstrap.cc:654] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:32.643741 24505 log.cc:824] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:32.645061 24506 tablet_bootstrap.cc:654] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:32.645905 24506 log.cc:824] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:32.646137 24505 tablet_bootstrap.cc:492] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94: No bootstrap required, opened a new log
I20260809 04:43:32.646350 24505 ts_tablet_manager.cc:1397] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260809 04:43:32.647825 24506 tablet_bootstrap.cc:492] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71: No bootstrap required, opened a new log
I20260809 04:43:32.648008 24506 ts_tablet_manager.cc:1397] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:43:32.650328 24505 raft_consensus.cc:348] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:32.650614 24505 raft_consensus.cc:374] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:32.650681 24505 raft_consensus.cc:729] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b4dc940ba00496ab450564bcd04df94, State: Initialized, Role: FOLLOWER
I20260809 04:43:32.651149 24505 consensus_queue.cc:260] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:32.651834 24491 heartbeater.cc:503] Master 127.5.222.254:44275 was elected leader, sending a full tablet report...
I20260809 04:43:32.651757 24506 raft_consensus.cc:348] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:32.652063 24506 raft_consensus.cc:374] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:32.652123 24506 raft_consensus.cc:729] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755f2395f65b48a5a32c2c7e9d534e71, State: Initialized, Role: FOLLOWER
I20260809 04:43:32.652213 24505 ts_tablet_manager.cc:1428] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260809 04:43:32.652522 24506 consensus_queue.cc:260] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:32.653146 24352 heartbeater.cc:503] Master 127.5.222.254:44275 was elected leader, sending a full tablet report...
I20260809 04:43:32.653481 24506 ts_tablet_manager.cc:1428] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
W20260809 04:43:32.680338 24206 tablet.cc:2369] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:43:32.824998 24492 tablet.cc:2369] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:43:32.825752 24509 raft_consensus.cc:482] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:32.825948 24509 raft_consensus.cc:504] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:32.827090 24509 leader_election.cc:288] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097), 986519d0eca94fc9ab1d2952a8e53b86 (127.5.222.193:38981)
I20260809 04:43:32.831957 24495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.827434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43097 (local address 127.5.222.195:44801)
0809 04:43:32.827582 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:32.827600 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:32.827712 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:32.828071 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:32.828074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:32.828098 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:32.828258 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.828266 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.829353 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.829356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.830655 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.830665 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.831506 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.831509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.831586 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.831631 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 04:43:32.831722 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 04:43:32.831799 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":27}
I20260809 04:43:32.832012 24501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.827520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:44801 (local address 127.5.222.194:43097)
0809 04:43:32.827663 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:32.827669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:32.827687 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:43:32.827880 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:32.827883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:32.827934 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:32.828013 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:32.828390 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.829218 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.830778 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.831388 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.831506 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.831560 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:32.831739 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 04:43:32.831862 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:43:32.832151 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.827931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:39685 (local address 127.5.222.193:38981)
0809 04:43:32.828203 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:32.828206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:32.828217 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:43:32.828278 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:32.828280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:32.828356 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:32.828471 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:32.828704 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.829431 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.830916 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.831555 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.831667 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.831833 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:32.831942 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 04:43:32.832013 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":76,"threads_started":1}
I20260809 04:43:32.832346 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.827746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:38981 (local address 127.5.222.195:39685)
0809 04:43:32.828086 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:32.828098 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:32.828177 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:32.828490 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:32.828492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:32.828507 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:32.828593 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.828598 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.829525 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.829528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.830790 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:32.830805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.831669 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.831673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:32.831821 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.831842 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:43:32.831912 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 04:43:32.831965 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":50,"thread_start_us":157,"threads_started":1}
I20260809 04:43:32.832645 24304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7130033ea9c4e58997d85f1e18d0929" candidate_uuid: "8b4dc940ba00496ab450564bcd04df94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "755f2395f65b48a5a32c2c7e9d534e71" is_pre_election: true
I20260809 04:43:32.832862 24159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7130033ea9c4e58997d85f1e18d0929" candidate_uuid: "8b4dc940ba00496ab450564bcd04df94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "986519d0eca94fc9ab1d2952a8e53b86" is_pre_election: true
I20260809 04:43:32.832996 24304 raft_consensus.cc:2454] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b4dc940ba00496ab450564bcd04df94 in term 0.
I20260809 04:43:32.833204 24159 raft_consensus.cc:2454] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b4dc940ba00496ab450564bcd04df94 in term 0.
I20260809 04:43:32.833508 24376 leader_election.cc:302] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755f2395f65b48a5a32c2c7e9d534e71, 8b4dc940ba00496ab450564bcd04df94; no voters:
I20260809 04:43:32.833828 24509 raft_consensus.cc:2781] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:43:32.833927 24509 raft_consensus.cc:482] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:43:32.834009 24509 raft_consensus.cc:3037] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:32.835058 24509 raft_consensus.cc:504] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:32.835399 24509 leader_election.cc:288] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [CANDIDATE]: Term 1 election: Requested vote from peers 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097), 986519d0eca94fc9ab1d2952a8e53b86 (127.5.222.193:38981)
I20260809 04:43:32.835817 24304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7130033ea9c4e58997d85f1e18d0929" candidate_uuid: "8b4dc940ba00496ab450564bcd04df94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "755f2395f65b48a5a32c2c7e9d534e71"
I20260809 04:43:32.835857 24159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7130033ea9c4e58997d85f1e18d0929" candidate_uuid: "8b4dc940ba00496ab450564bcd04df94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "986519d0eca94fc9ab1d2952a8e53b86"
I20260809 04:43:32.835983 24159 raft_consensus.cc:3037] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:32.835961 24304 raft_consensus.cc:3037] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:32.837007 24159 raft_consensus.cc:2454] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b4dc940ba00496ab450564bcd04df94 in term 1.
I20260809 04:43:32.837049 24304 raft_consensus.cc:2454] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b4dc940ba00496ab450564bcd04df94 in term 1.
I20260809 04:43:32.837298 24376 leader_election.cc:302] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b4dc940ba00496ab450564bcd04df94, 986519d0eca94fc9ab1d2952a8e53b86; no voters:
I20260809 04:43:32.837503 24509 raft_consensus.cc:2781] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:32.837726 24509 raft_consensus.cc:686] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 1 LEADER]: Becoming Leader. State: Replica: 8b4dc940ba00496ab450564bcd04df94, State: Running, Role: LEADER
I20260809 04:43:32.838060 24509 consensus_queue.cc:237] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:32.841066 24008 catalog_manager.cc:5495] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195). New cstate: current_term: 1 leader_uuid: "8b4dc940ba00496ab450564bcd04df94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } health_report { overall_health: UNKNOWN } } }
I20260809 04:43:32.863175 24503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.859620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47600 (local address 127.5.222.195:40233)
0809 04:43:32.859825 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:32.859829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:32.859843 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:43:32.859971 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:32.859974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:32.860026 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:32.860110 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:32.860337 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.860878 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.861866 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:32.862089 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:32.862173 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:32.862454 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:32.862837 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:32.862927 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:32.863003 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 04:43:32.863054 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 04:43:32.869014 24304 raft_consensus.cc:1261] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71 [term 1 FOLLOWER]: Refusing update from remote peer 8b4dc940ba00496ab450564bcd04df94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:32.869012 24159 raft_consensus.cc:1261] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Refusing update from remote peer 8b4dc940ba00496ab450564bcd04df94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:32.869766 24509 consensus_queue.cc:1035] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:32.869966 24513 consensus_queue.cc:1035] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 04:43:32.875460 24353 tablet.cc:2369] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:43:32.875931 24515 mvcc.cc:205] Tried to move back new op lower bound from 7316482510303850496 to 7316482510187118592. Current Snapshot: MvccSnapshot[applied={T|T < 7316482510303850496}]
I20260809 04:43:32.876262 24516 mvcc.cc:205] Tried to move back new op lower bound from 7316482510303850496 to 7316482510187118592. Current Snapshot: MvccSnapshot[applied={T|T < 7316482510303850496}]
I20260809 04:43:32.885296 24006 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:32.885473 24006 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:32.887979 24006 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Sending DeleteTablet for 3 replicas of tablet d7130033ea9c4e58997d85f1e18d0929
I20260809 04:43:32.888715 24424 tablet_service.cc:1514] Processing DeleteTablet for tablet d7130033ea9c4e58997d85f1e18d0929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:32 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:47590
I20260809 04:43:32.888821 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 24048
I20260809 04:43:32.888880 24284 tablet_service.cc:1514] Processing DeleteTablet for tablet d7130033ea9c4e58997d85f1e18d0929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:32 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:46996
I20260809 04:43:32.891906 24527 tablet_replica.cc:330] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71: stopping tablet replica
I20260809 04:43:32.892316 24527 raft_consensus.cc:2229] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:32.892714 24527 raft_consensus.cc:2258] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:32.894167 24527 ts_tablet_manager.cc:1905] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:32.896361 24527 ts_tablet_manager.cc:1918] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:43:32.896482 24527 log.cc:1196] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/d7130033ea9c4e58997d85f1e18d0929
I20260809 04:43:32.896802 24527 ts_tablet_manager.cc:1939] T d7130033ea9c4e58997d85f1e18d0929 P 755f2395f65b48a5a32c2c7e9d534e71: Deleting consensus metadata
I20260809 04:43:32.898063 24526 tablet_replica.cc:330] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94: stopping tablet replica
I20260809 04:43:32.898097 23993 catalog_manager.cc:4841] TS 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097): tablet d7130033ea9c4e58997d85f1e18d0929 (table test-table [id=84fb6049d5fc431098f7d41b2498ac61]) successfully deleted
I20260809 04:43:32.898396 24526 raft_consensus.cc:2229] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:43:32.898737 24526 raft_consensus.cc:2258] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 04:43:32.899246 23993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:38981 (error 108)
I20260809 04:43:32.899392 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 24210
I20260809 04:43:32.902328 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.901897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:38981 (local address 127.0.0.1:59896)
0809 04:43:32.902118 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:32.902213 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260809 04:43:32.902644 23993 catalog_manager.cc:4584] TS 986519d0eca94fc9ab1d2952a8e53b86 (127.5.222.193:38981): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d7130033ea9c4e58997d85f1e18d0929: Network error: Client connection negotiation failed: client connection to 127.5.222.193:38981: connect: Connection refused (error 111)
I20260809 04:43:32.904120 24526 ts_tablet_manager.cc:1905] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:32.906841 24526 ts_tablet_manager.cc:1918] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:43:32.906953 24526 log.cc:1196] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/d7130033ea9c4e58997d85f1e18d0929
I20260809 04:43:32.907299 24526 ts_tablet_manager.cc:1939] T d7130033ea9c4e58997d85f1e18d0929 P 8b4dc940ba00496ab450564bcd04df94: Deleting consensus metadata
I20260809 04:43:32.907617 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 24356
I20260809 04:43:32.913877 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 23973
I20260809 04:43:52.922585 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:44275
--webserver_interface=127.5.222.254
--webserver_port=33447
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:44275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:53.034040 24529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:53.034304 24529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:53.034344 24529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:53.038424 24529 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 04:43:53.038478 24529 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 04:43:53.038496 24529 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 04:43:53.038527 24529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:43:53.038545 24529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:53.038602 24529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:53.038633 24529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:43:53.038658 24529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:43:53.043783 24529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:44275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:44275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=33447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:43:53.044780 24529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:53.045864 24529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:53.054286 24534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:53.054301 24535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:53.054296 24537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:53.054358 24529 server_base.cc:1034] running on GCE node
I20260809 04:43:53.055122 24529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:53.056232 24529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:53.057412 24529 hybrid_clock.cc:648] HybridClock initialized: now 1786250633057393 us; error 31 us; skew 500 ppm
Aug 09 04:43:53 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250633, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:53.060097 24529 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:43:53.060997 24529 webserver.cc:460] Webserver started at http://127.5.222.254:33447/ using document root <none> and password file <none>
I20260809 04:43:53.061625 24529 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:53.061718 24529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:53.065542 24529 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 04:43:53.067734 24544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:53.069039 24529 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:43:53.069204 24529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "14d3dd5b39a748359814f3da25173f6d"
format_stamp: "Formatted at 2026-08-09 04:43:10 on dist-test-slave-vxj2"
server_key: "d0ddd280b7fab73a19bef23f18fb310d"
server_key_iv: "167181c79ad6d027165913fde2c2cb8b"
server_key_version: "encryptionkey@0"
I20260809 04:43:53.069813 24529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:53.099292 24529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:53.102535 24529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:53.102799 24529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:53.110466 24529 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:44275
I20260809 04:43:53.110867 24599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:44275 every 8 connection(s)
I20260809 04:43:53.111606 24529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:43:53.115970 24600 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:43:53.117719 24600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Bootstrap starting.
I20260809 04:43:53.120198 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 24529
I20260809 04:43:53.120728 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:38981
--local_ip_for_outbound_sockets=127.5.222.193
--tserver_master_addrs=127.5.222.254:44275
--webserver_port=44621
--webserver_interface=127.5.222.193
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:43:53.136268 24600 log.cc:824] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:53.148031 24600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:43:53.148433 24600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Bootstrap complete.
I20260809 04:43:53.152555 24600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } }
I20260809 04:43:53.152947 24600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d3dd5b39a748359814f3da25173f6d, State: Initialized, Role: FOLLOWER
I20260809 04:43:53.153470 24600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } }
I20260809 04:43:53.153592 24600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:43:53.153682 24600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:43:53.153806 24600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 2 FOLLOWER]: Advancing to term 3
I20260809 04:43:53.155763 24600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } }
I20260809 04:43:53.156095 24600 leader_election.cc:302] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14d3dd5b39a748359814f3da25173f6d; no voters:
I20260809 04:43:53.156432 24600 leader_election.cc:288] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 04:43:53.156517 24604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 3 FOLLOWER]: Leader election won for term 3
I20260809 04:43:53.156745 24604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [term 3 LEADER]: Becoming Leader. State: Replica: 14d3dd5b39a748359814f3da25173f6d, State: Running, Role: LEADER
I20260809 04:43:53.157102 24604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } }
I20260809 04:43:53.157701 24600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:43:53.158618 24606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14d3dd5b39a748359814f3da25173f6d. Latest consensus state: current_term: 3 leader_uuid: "14d3dd5b39a748359814f3da25173f6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } } }
I20260809 04:43:53.158751 24606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:53.158675 24605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "14d3dd5b39a748359814f3da25173f6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d3dd5b39a748359814f3da25173f6d" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 44275 } } }
I20260809 04:43:53.158797 24605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:53.159193 24613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:43:53.164438 24613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:43:53.165217 24613 catalog_manager.cc:1259] Loaded cluster ID: 2b1a82ce1457403b91e6bfba1a63733a
I20260809 04:43:53.165290 24613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:43:53.167909 24613 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:43:53.168670 24613 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Loaded TSK: 0
I20260809 04:43:53.168757 24613 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 14d3dd5b39a748359814f3da25173f6d: Loaded TSK: 1
F20260809 04:43:53.169590 24613 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2ffec1d9ad google::LogMessage::Fail() at ??:0
@ 0x7f2ffec2088a google::LogMessage::SendToLog() at ??:0
@ 0x7f2ffec1d4de google::LogMessage::Flush() at ??:0
@ 0x7f2ffec1ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f30018e118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f30018d9f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f30095827bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f300955c99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f300955d178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f300959edf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f30095bd82b std::function<>::operator()() at ??:0
@ 0x7f300955cb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f300955df3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f300955a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f300959e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f3000e6f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3000e6fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f3000e713f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f3000e60512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f30024a46db start_thread at ??:0
@ 0x7f2ffd71b61f clone at ??:0
W20260809 04:43:53.262748 24602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:53.262976 24602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:53.263018 24602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:53.267361 24602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:53.267407 24602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:53.267491 24602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:43:53.273264 24602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:38981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=44621
--tserver_master_addrs=127.5.222.254:44275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:43:53.274286 24602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:53.275255 24602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:53.285108 24651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:53.285108 24653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:53.285394 24602 server_base.cc:1034] running on GCE node
W20260809 04:43:53.285225 24650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:53.286060 24602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:53.286892 24602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:53.288107 24602 hybrid_clock.cc:648] HybridClock initialized: now 1786250633288083 us; error 32 us; skew 500 ppm
Aug 09 04:43:53 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250633, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:53.291412 24602 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:43:53.292259 24602 webserver.cc:460] Webserver started at http://127.5.222.193:44621/ using document root <none> and password file <none>
I20260809 04:43:53.292780 24602 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:53.292840 24602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:53.296645 24602 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:43:53.299139 24660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:53.300416 24602 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:43:53.300554 24602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "986519d0eca94fc9ab1d2952a8e53b86"
format_stamp: "Formatted at 2026-08-09 04:43:10 on dist-test-slave-vxj2"
server_key: "87b5ea09dc3f202ab2fad65c62945d56"
server_key_iv: "0c0f25f82bb10c511466d53a5781de7d"
server_key_version: "encryptionkey@0"
I20260809 04:43:53.301088 24602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:53.333181 24602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:53.336952 24602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:53.337250 24602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:53.337931 24602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:53.339210 24668 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 04:43:53.344846 24602 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 04:43:53.344949 24602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 04:43:53.345052 24602 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 04:43:53.346803 24602 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 04:43:53.346881 24602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 04:43:53.347172 24668 tablet_bootstrap.cc:492] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86: Bootstrap starting.
I20260809 04:43:53.357777 24602 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:38981
I20260809 04:43:53.358719 24602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:43:53.359459 24779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:38981 every 8 connection(s)
I20260809 04:43:53.368335 24668 log.cc:824] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:53.368815 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 24602
I20260809 04:43:53.369386 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:43097
--local_ip_for_outbound_sockets=127.5.222.194
--tserver_master_addrs=127.5.222.254:44275
--webserver_port=33161
--webserver_interface=127.5.222.194
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:43:53.375831 24668 tablet_bootstrap.cc:492] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:43:53.376228 24668 tablet_bootstrap.cc:492] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86: Bootstrap complete.
I20260809 04:43:53.377384 24668 ts_tablet_manager.cc:1397] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86: Time spent bootstrapping tablet: real 0.030s user 0.011s sys 0.017s
I20260809 04:43:53.379719 24668 raft_consensus.cc:348] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:53.380224 24668 raft_consensus.cc:729] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 986519d0eca94fc9ab1d2952a8e53b86, State: Initialized, Role: FOLLOWER
I20260809 04:43:53.380782 24668 consensus_queue.cc:260] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:53.382025 24668 ts_tablet_manager.cc:1428] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
W20260809 04:43:53.431916 24782 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:43:53.360985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:54499)
0809 04:43:53.361633 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:53.361691 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:53.362840 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:53.431137 (+ 68297us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":170,"threads_started":1}
W20260809 04:43:53.432556 24780 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:44275 (0 consecutive failures): Network error: Failed to ping master at 127.5.222.254:44275: Client connection negotiation failed: client connection to 127.5.222.254:44275: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 04:43:53.433342 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:53.433055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:41965)
0809 04:43:53.433211 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:53.433287 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 04:43:53.434067 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:53.433846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:37603)
0809 04:43:53.433968 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:53.434018 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 04:43:53.434270 24780 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:43:53.501365 24784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:53.501641 24784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:53.501751 24784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:53.505996 24784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:53.506083 24784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:53.506170 24784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:43:53.511497 24784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:43097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=33161
--tserver_master_addrs=127.5.222.254:44275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:43:53.512593 24784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:53.513691 24784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:53.523694 24790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:53.523723 24791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:53.523743 24784 server_base.cc:1034] running on GCE node
W20260809 04:43:53.523694 24793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:53.524394 24784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:53.525015 24784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:53.526201 24784 hybrid_clock.cc:648] HybridClock initialized: now 1786250633526178 us; error 35 us; skew 500 ppm
Aug 09 04:43:53 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250633, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:53.528926 24784 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:43:53.529860 24784 webserver.cc:460] Webserver started at http://127.5.222.194:33161/ using document root <none> and password file <none>
I20260809 04:43:53.530445 24784 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:53.530560 24784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:53.534459 24784 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 04:43:53.536535 24800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:53.537971 24784 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:43:53.538122 24784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "755f2395f65b48a5a32c2c7e9d534e71"
format_stamp: "Formatted at 2026-08-09 04:43:11 on dist-test-slave-vxj2"
server_key: "b3f38d156b81de20f06f873855552a45"
server_key_iv: "3323938608c4d5a923a5c0c5f7d3f762"
server_key_version: "encryptionkey@0"
I20260809 04:43:53.538703 24784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:53.560874 24784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:53.564100 24784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:53.564303 24784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:53.564908 24784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:53.565802 24784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:53.565876 24784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:53.565943 24784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:53.565990 24784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:53.575284 24784 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:43097
I20260809 04:43:53.575311 24918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:43097 every 8 connection(s)
I20260809 04:43:53.576294 24784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:43:53.579015 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 24784
I20260809 04:43:53.579594 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:40233
--local_ip_for_outbound_sockets=127.5.222.195
--tserver_master_addrs=127.5.222.254:44275
--webserver_port=44117
--webserver_interface=127.5.222.195
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:46771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:43:53.580657 24921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:53.578517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:46265)
0809 04:43:53.579049 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:53.579657 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":123,"threads_started":1}
W20260809 04:43:53.581136 24919 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:44275 (0 consecutive failures): Network error: Failed to ping master at 127.5.222.254:44275: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
I20260809 04:43:53.582086 24921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:53.581821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:55963)
0809 04:43:53.581952 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:53.582004 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 04:43:53.582760 24921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:53.582541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:55523)
0809 04:43:53.582651 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:53.582699 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 04:43:53.583000 24919 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:43:53.690115 24922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:53.690410 24922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:53.690486 24922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:53.694506 24922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:53.694591 24922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:53.694679 24922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:43:53.699705 24922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:40233
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=44117
--tserver_master_addrs=127.5.222.254:44275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:43:53.701017 24922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:53.702036 24922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:53.711710 24927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:53.711717 24928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:53.711848 24922 server_base.cc:1034] running on GCE node
W20260809 04:43:53.712042 24930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:53.712423 24922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:53.713032 24922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:53.714226 24922 hybrid_clock.cc:648] HybridClock initialized: now 1786250633714181 us; error 44 us; skew 500 ppm
Aug 09 04:43:53 dist-test-slave-vxj2 krb5kdc[23394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250633, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:53.716915 24922 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:43:53.717767 24922 webserver.cc:460] Webserver started at http://127.5.222.195:44117/ using document root <none> and password file <none>
I20260809 04:43:53.718279 24922 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:53.718338 24922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:53.722033 24922 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:43:53.724252 24937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:53.725629 24922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:43:53.725811 24922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "8b4dc940ba00496ab450564bcd04df94"
format_stamp: "Formatted at 2026-08-09 04:43:11 on dist-test-slave-vxj2"
server_key: "74d3c6f890ac7c300c906eece25fe8e6"
server_key_iv: "121e7718c8fcd92e4ac970c9bfdc94fe"
server_key_version: "encryptionkey@0"
I20260809 04:43:53.726339 24922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:53.755708 24922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:53.758675 24922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:53.758893 24922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:53.759493 24922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:53.760337 24922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:53.760385 24922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:53.760455 24922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:53.760499 24922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:53.769680 24922 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:40233
I20260809 04:43:53.769713 25054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:40233 every 8 connection(s)
I20260809 04:43:53.770690 24922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:43:53.774361 25058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:53.772780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:43571)
0809 04:43:53.773231 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:53.773659 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":106,"threads_started":1}
W20260809 04:43:53.774796 25055 heartbeater.cc:644] Failed to heartbeat to 127.5.222.254:44275 (0 consecutive failures): Network error: Failed to ping master at 127.5.222.254:44275: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
I20260809 04:43:53.775349 25058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:53.775147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:39523)
0809 04:43:53.775250 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:53.775295 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 04:43:53.775894 25058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:53.775718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:33571)
0809 04:43:53.775808 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:53.775848 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 04:43:53.776116 25055 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 04:43:53.777459 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 24922
I20260809 04:43:54.435369 25060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:54.434947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:57041)
0809 04:43:54.435195 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:54.435252 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":78,"threads_started":1}
I20260809 04:43:54.583930 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:54.583541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:43395)
0809 04:43:54.583770 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:54.583826 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":70,"threads_started":1}
I20260809 04:43:54.777407 25062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:54.776913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:41501)
0809 04:43:54.777230 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:54.777293 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":123,"threads_started":1}
I20260809 04:43:54.895787 25063 raft_consensus.cc:482] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:54.896049 25063 raft_consensus.cc:504] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:54.897367 25063 leader_election.cc:288] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097), 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233)
Aug 09 04:43:54 dist-test-slave-vxj2 krb5kdc[23394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250633, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.194@KRBTEST.COM
Aug 09 04:43:54 dist-test-slave-vxj2 krb5kdc[23394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250633, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.195@KRBTEST.COM
I20260809 04:43:54.911582 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:54.897840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:54483 (local address 127.5.222.195:40233)
0809 04:43:54.898150 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:54.898175 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:54.898217 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:43:54.898884 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:54.898889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:54.899191 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:54.899481 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:54.900684 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:54.901721 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:54.903010 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:54.903579 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:54.903732 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:54.906915 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:54.906923 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:54.906941 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:54.906982 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:54.909381 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:54.909897 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:54.909903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:54.909908 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:54.910040 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:54.910334 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:54.910336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:54.910337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:54.910666 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:54.910811 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:54.911144 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 04:43:54.911389 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
I20260809 04:43:54.911585 25064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:54.897601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:40233 (local address 127.5.222.193:54483)
0809 04:43:54.898086 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:54.898099 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:54.898196 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:54.899836 (+ 1640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:54.899838 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:54.900117 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:54.900490 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:54.900505 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:54.901884 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:54.901888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:54.902883 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:54.902893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:54.903703 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:54.903717 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:54.903892 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:54.904630 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:54.904642 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:54.906690 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:54.909515 (+ 2825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:54.909518 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:54.909520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:54.909758 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:54.910139 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:54.910142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:54.910144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:54.910229 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:54.910782 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:54.910785 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:54.910888 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 04:43:54.911012 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 04:43:54.911348 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":61,"spinlock_wait_cycles":768,"thread_start_us":64,"threads_started":1}
I20260809 04:43:54.911610 25065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:54.897682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:57655 (local address 127.5.222.194:43097)
0809 04:43:54.898017 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:54.898044 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:54.898093 (+ 49us) server_negotiation.cc:407] Connection header received
0809 04:43:54.898827 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:54.898834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:54.899191 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:54.899461 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:54.900677 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:54.901731 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:54.903012 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:54.903609 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:54.903786 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:54.906590 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:54.906595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:54.906606 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:54.906634 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:54.908851 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:54.909398 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:54.909406 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:54.909412 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:54.909569 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:54.909943 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:54.909947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:54.909949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:54.910469 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:54.910609 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:54.911126 (+ 517us) server_negotiation.cc:299] Negotiation successful
0809 04:43:54.911420 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":104,"threads_started":1}
I20260809 04:43:54.911594 25060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:54.897506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43097 (local address 127.5.222.193:57655)
0809 04:43:54.897715 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:54.897731 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:54.897842 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:54.899539 (+ 1697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:54.899545 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:54.900037 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:54.900491 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:54.900504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:54.901891 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:54.901893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:54.902881 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:54.902891 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:54.903753 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:54.903757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:54.903899 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:54.904578 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:54.904594 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:54.906354 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:54.908983 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:54.908989 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:54.908999 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:54.909286 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:54.909670 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:54.909674 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:54.909695 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:54.909841 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:54.910581 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:54.910587 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:54.910819 (+ 232us) client_negotiation.cc:769] Sending connection context
0809 04:43:54.911000 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 04:43:54.911433 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":280}
W20260809 04:43:54.914009 24663 leader_election.cc:341] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097): Not found: Tablet not found: d7130033ea9c4e58997d85f1e18d0929
W20260809 04:43:54.914172 24664 leader_election.cc:341] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233): Not found: Tablet not found: d7130033ea9c4e58997d85f1e18d0929
I20260809 04:43:54.914248 24664 leader_election.cc:302] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 986519d0eca94fc9ab1d2952a8e53b86; no voters: 755f2395f65b48a5a32c2c7e9d534e71, 8b4dc940ba00496ab450564bcd04df94
I20260809 04:43:54.914462 25063 raft_consensus.cc:2726] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:43:55.436599 25067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:55.436189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:59735)
0809 04:43:55.436442 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:55.436502 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":82,"threads_started":1}
I20260809 04:43:55.585074 25068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:55.584660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:53875)
0809 04:43:55.584914 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:55.584982 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":96,"threads_started":1}
I20260809 04:43:55.778555 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:55.778146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:55255)
0809 04:43:55.778422 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:55.778475 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":111,"threads_started":1}
I20260809 04:43:56.437774 25070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:56.437380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:46101)
0809 04:43:56.437624 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:56.437678 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":77,"threads_started":1}
I20260809 04:43:56.510594 25071 raft_consensus.cc:482] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:56.510736 25071 raft_consensus.cc:504] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:56.511212 25071 leader_election.cc:288] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097), 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233)
W20260809 04:43:56.512377 24664 leader_election.cc:341] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233): Not found: Tablet not found: d7130033ea9c4e58997d85f1e18d0929
W20260809 04:43:56.512508 24663 leader_election.cc:341] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097): Not found: Tablet not found: d7130033ea9c4e58997d85f1e18d0929
I20260809 04:43:56.512598 24663 leader_election.cc:302] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 986519d0eca94fc9ab1d2952a8e53b86; no voters: 755f2395f65b48a5a32c2c7e9d534e71, 8b4dc940ba00496ab450564bcd04df94
I20260809 04:43:56.512790 25071 raft_consensus.cc:2726] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:43:56.586336 25072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:56.585899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:47787)
0809 04:43:56.586166 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:56.586228 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":78,"threads_started":1}
I20260809 04:43:56.779714 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:56.779300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:55807)
0809 04:43:56.779553 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:56.779608 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":80,"threads_started":1}
I20260809 04:43:57.439055 25074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:57.438561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:48787)
0809 04:43:57.438863 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:57.438954 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":81,"threads_started":1}
I20260809 04:43:57.587587 25075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:57.587162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:46343)
0809 04:43:57.587426 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:57.587489 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
I20260809 04:43:57.781368 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:57.780787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:34977)
0809 04:43:57.781141 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:57.781232 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":114,"threads_started":1}
I20260809 04:43:58.440614 25077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:58.440096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:40635)
0809 04:43:58.440453 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:58.440518 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":93,"threads_started":1}
I20260809 04:43:58.577351 25078 raft_consensus.cc:482] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:58.577566 25078 raft_consensus.cc:504] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:43:58.578122 25078 leader_election.cc:288] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097), 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233)
W20260809 04:43:58.578905 24664 leader_election.cc:341] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233): Not found: Tablet not found: d7130033ea9c4e58997d85f1e18d0929
W20260809 04:43:58.579041 24663 leader_election.cc:341] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097): Not found: Tablet not found: d7130033ea9c4e58997d85f1e18d0929
I20260809 04:43:58.579102 24663 leader_election.cc:302] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 986519d0eca94fc9ab1d2952a8e53b86; no voters: 755f2395f65b48a5a32c2c7e9d534e71, 8b4dc940ba00496ab450564bcd04df94
I20260809 04:43:58.579250 25078 raft_consensus.cc:2726] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:43:58.588852 25079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:58.588407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:53221)
0809 04:43:58.588671 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:58.588738 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":108,"threads_started":1}
I20260809 04:43:58.782548 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:58.782141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:46375)
0809 04:43:58.782401 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:58.782460 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
I20260809 04:43:59.441969 25081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:59.441472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:38641)
0809 04:43:59.441737 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:59.441795 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":81,"threads_started":1}
I20260809 04:43:59.590118 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:59.589714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:39011)
0809 04:43:59.589965 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:59.590023 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
I20260809 04:43:59.783720 25083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:59.783310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:52387)
0809 04:43:59.783573 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:59.783630 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":75,"threads_started":1}
I20260809 04:44:00.443316 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:00.442743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:52481)
0809 04:44:00.443105 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:00.443192 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":97,"threads_started":1}
I20260809 04:44:00.591421 25085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:00.590958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:36415)
0809 04:44:00.591261 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:00.591326 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":75,"threads_started":1}
I20260809 04:44:00.785204 25086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:00.784605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:55351)
0809 04:44:00.784992 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:00.785057 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":149,"threads_started":1}
I20260809 04:44:01.205667 25087 raft_consensus.cc:482] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:01.205809 25087 raft_consensus.cc:504] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:44:01.206274 25087 leader_election.cc:288] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097), 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233)
W20260809 04:44:01.207216 24663 leader_election.cc:341] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097): Not found: Tablet not found: d7130033ea9c4e58997d85f1e18d0929
W20260809 04:44:01.207341 24664 leader_election.cc:341] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233): Not found: Tablet not found: d7130033ea9c4e58997d85f1e18d0929
I20260809 04:44:01.207409 24664 leader_election.cc:302] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 986519d0eca94fc9ab1d2952a8e53b86; no voters: 755f2395f65b48a5a32c2c7e9d534e71, 8b4dc940ba00496ab450564bcd04df94
I20260809 04:44:01.207549 25087 raft_consensus.cc:2726] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:44:01.444665 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:01.444207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:58065)
0809 04:44:01.444499 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:01.444566 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":91,"threads_started":1}
I20260809 04:44:01.592891 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:01.592445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:54221)
0809 04:44:01.592725 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:01.592791 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
I20260809 04:44:01.786454 25090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:01.785987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:40011)
0809 04:44:01.786287 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:01.786342 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":104,"threads_started":1}
I20260809 04:44:02.446019 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:02.445563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:50673)
0809 04:44:02.445846 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:02.445910 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
I20260809 04:44:02.594172 25092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:02.593759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:50881)
0809 04:44:02.594019 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:02.594076 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":76,"threads_started":1}
I20260809 04:44:02.787673 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:02.787237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:40093)
0809 04:44:02.787503 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:02.787581 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":90,"threads_started":1}
I20260809 04:44:03.447455 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:03.446979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:54693)
0809 04:44:03.447289 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:03.447349 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":83,"threads_started":1}
I20260809 04:44:03.595526 25095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:03.595016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:57745)
0809 04:44:03.595349 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:03.595422 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":78,"threads_started":1}
I20260809 04:44:03.788889 25096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:03.788506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:53833)
0809 04:44:03.788744 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:03.788798 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":72,"threads_started":1}
I20260809 04:44:04.448648 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.448210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:43653)
0809 04:44:04.448488 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:04.448549 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":76,"threads_started":1}
I20260809 04:44:04.596807 25098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.596293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:46383)
0809 04:44:04.596603 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:04.596670 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":72,"threads_started":1}
I20260809 04:44:04.790047 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.789628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:49723)
0809 04:44:04.789897 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:04.789955 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":124,"threads_started":1}
I20260809 04:44:05.450089 25100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:05.449611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:47399)
0809 04:44:05.449896 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:05.449967 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":88,"threads_started":1}
I20260809 04:44:05.598043 25101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:05.597621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:38117)
0809 04:44:05.597880 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:05.597940 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":78,"threads_started":1}
I20260809 04:44:05.791293 25102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:05.790894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:56559)
0809 04:44:05.791137 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:05.791195 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
I20260809 04:44:06.451339 25103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:06.450876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:38021)
0809 04:44:06.451176 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:06.451236 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":92,"threads_started":1}
I20260809 04:44:06.599435 25104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:06.598950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:60255)
0809 04:44:06.599244 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:06.599315 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":92,"threads_started":1}
I20260809 04:44:06.792469 25106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:06.792066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:59845)
0809 04:44:06.792308 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:06.792364 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":79,"threads_started":1}
I20260809 04:44:07.001873 25107 raft_consensus.cc:482] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:07.002027 25107 raft_consensus.cc:504] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755f2395f65b48a5a32c2c7e9d534e71" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43097 } } peers { permanent_uuid: "8b4dc940ba00496ab450564bcd04df94" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 40233 } } peers { permanent_uuid: "986519d0eca94fc9ab1d2952a8e53b86" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38981 } }
I20260809 04:44:07.002511 25107 leader_election.cc:288] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097), 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233)
W20260809 04:44:07.003441 24663 leader_election.cc:341] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 755f2395f65b48a5a32c2c7e9d534e71 (127.5.222.194:43097): Not found: Tablet not found: d7130033ea9c4e58997d85f1e18d0929
W20260809 04:44:07.003545 24664 leader_election.cc:341] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8b4dc940ba00496ab450564bcd04df94 (127.5.222.195:40233): Not found: Tablet not found: d7130033ea9c4e58997d85f1e18d0929
I20260809 04:44:07.003639 24664 leader_election.cc:302] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 986519d0eca94fc9ab1d2952a8e53b86; no voters: 755f2395f65b48a5a32c2c7e9d534e71, 8b4dc940ba00496ab450564bcd04df94
I20260809 04:44:07.003816 25107 raft_consensus.cc:2726] T d7130033ea9c4e58997d85f1e18d0929 P 986519d0eca94fc9ab1d2952a8e53b86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:44:07.452796 25108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.452234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:57975)
0809 04:44:07.452579 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.452676 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":138,"threads_started":1}
I20260809 04:44:07.601009 25109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.600395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:43545)
0809 04:44:07.600787 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.600861 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":141,"threads_started":1}
I20260809 04:44:07.793628 25110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.793218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:55477)
0809 04:44:07.793487 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.793540 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":84,"threads_started":1}
I20260809 04:44:08.454159 25111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:08.453726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.193:37423)
0809 04:44:08.453994 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:08.454058 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":75,"threads_started":1}
I20260809 04:44:08.602540 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:08.602020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.194:46505)
0809 04:44:08.602374 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:08.602440 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":107,"threads_started":1}
I20260809 04:44:08.788332 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 24602
I20260809 04:44:08.794883 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:08.794485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:44275 (local address 127.5.222.195:53901)
0809 04:44:08.794739 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:08.794794 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.254:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":92,"threads_started":1}
I20260809 04:44:08.797210 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 24784
I20260809 04:44:08.803824 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 24922
2026-08-09T04:44:08Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59668 ms)
[----------] 30 tests from SecurityITest (245305 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:08 dist-test-slave-vxj2 krb5kdc[25116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:08 dist-test-slave-vxj2 krb5kdc[25116](info): set up 2 sockets
Aug 09 04:44:08 dist-test-slave-vxj2 krb5kdc[25116](info): commencing operation
krb5kdc: starting...
W20260809 04:44:09.909468 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:09 dist-test-slave-vxj2 krb5kdc[25116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:44:10.036193 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:41565
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:41565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:10.147032 25129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:10.147336 25129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:10.147406 25129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:10.151472 25129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:10.151559 25129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:10.151599 25129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:10.151655 25129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:10.151693 25129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:10.156841 25129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:41565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:41565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:10.157943 25129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:10.158965 25129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:10.167743 25137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:10.167879 25136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:10.167977 25129 server_base.cc:1034] running on GCE node
W20260809 04:44:10.167913 25139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:10.168695 25129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:10.169783 25129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:10.170980 25129 hybrid_clock.cc:648] HybridClock initialized: now 1786250650170952 us; error 42 us; skew 500 ppm
Aug 09 04:44:10 dist-test-slave-vxj2 krb5kdc[25116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250650, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:10.173755 25129 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:44:10.174674 25129 webserver.cc:460] Webserver started at http://127.5.222.254:45159/ using document root <none> and password file <none>
I20260809 04:44:10.175243 25129 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:10.175335 25129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:10.175573 25129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:10.177670 25129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "25842abfa1ea4de898286fd4adf622f1"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-vxj2"
server_key: "314eef508d0b546c90206ad73099a15b"
server_key_iv: "976a1b0144f2284b416b7413a96ad3cf"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.178253 25129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "25842abfa1ea4de898286fd4adf622f1"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-vxj2"
server_key: "314eef508d0b546c90206ad73099a15b"
server_key_iv: "976a1b0144f2284b416b7413a96ad3cf"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.182199 25129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:44:10.184505 25147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:10.185796 25129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:44:10.185961 25129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "25842abfa1ea4de898286fd4adf622f1"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-vxj2"
server_key: "314eef508d0b546c90206ad73099a15b"
server_key_iv: "976a1b0144f2284b416b7413a96ad3cf"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.186079 25129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:10.196830 25129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:10.199739 25129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:10.199950 25129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:10.207450 25129 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:41565
I20260809 04:44:10.207479 25206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:41565 every 8 connection(s)
I20260809 04:44:10.208380 25129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:10.211074 25207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:10.212769 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 25129
I20260809 04:44:10.212895 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:10.213222 6011 external_mini_cluster.cc:1428] Setting key 1b64c57aa7217e46ba0a40fd1ab38b71
I20260809 04:44:10.217654 25207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1: Bootstrap starting.
I20260809 04:44:10.220082 25207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:10.221061 25207 log.cc:824] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:44:10 dist-test-slave-vxj2 krb5kdc[25116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:10.223589 25207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1: No bootstrap required, opened a new log
I20260809 04:44:10.228150 25210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:10.214663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40012 (local address 127.5.222.254:41565)
0809 04:44:10.215128 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:10.215142 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:10.215177 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:44:10.215841 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:10.215859 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:10.216167 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:10.216505 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:10.217356 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.218380 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.219304 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.219777 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.219950 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:10.223191 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:10.223196 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:10.223209 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:10.223235 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:10.225047 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:10.225657 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:10.225661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:10.225665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:10.225777 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:10.226198 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:10.226200 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:10.226201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:10.226587 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:10.226728 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:10.227113 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 04:44:10.227420 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":140,"threads_started":1}
I20260809 04:44:10.228578 25207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25842abfa1ea4de898286fd4adf622f1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41565 } }
I20260809 04:44:10.228816 25207 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:10.228928 25207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25842abfa1ea4de898286fd4adf622f1, State: Initialized, Role: FOLLOWER
I20260809 04:44:10.229511 25207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25842abfa1ea4de898286fd4adf622f1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41565 } }
I20260809 04:44:10.229703 25207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:10.229774 25207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:10.229878 25207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:10.230988 25207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25842abfa1ea4de898286fd4adf622f1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41565 } }
I20260809 04:44:10.231312 25207 leader_election.cc:302] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25842abfa1ea4de898286fd4adf622f1; no voters:
I20260809 04:44:10.231573 25207 leader_election.cc:288] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:10.231714 25212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:10.231994 25212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [term 1 LEADER]: Becoming Leader. State: Replica: 25842abfa1ea4de898286fd4adf622f1, State: Running, Role: LEADER
I20260809 04:44:10.232366 25212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25842abfa1ea4de898286fd4adf622f1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41565 } }
I20260809 04:44:10.232864 25207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:10.234195 25213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25842abfa1ea4de898286fd4adf622f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25842abfa1ea4de898286fd4adf622f1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41565 } } }
I20260809 04:44:10.234342 25213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:10.234210 25214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25842abfa1ea4de898286fd4adf622f1. Latest consensus state: current_term: 1 leader_uuid: "25842abfa1ea4de898286fd4adf622f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25842abfa1ea4de898286fd4adf622f1" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 41565 } } }
I20260809 04:44:10.234412 25214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:10.235159 25222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:10.238039 25222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:10.243933 25222 catalog_manager.cc:1347] Generated new cluster ID: e3d47c4a02774b929482916fc6ebf486
I20260809 04:44:10.244005 25222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:10.286865 25222 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:10.288233 25222 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:10.294808 25222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1: Generated new TSK 0
I20260809 04:44:10.295544 25222 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:44:10.323642 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41565
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:10.436093 25235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:10.436378 25235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:10.436456 25235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:10.440464 25235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:10.440562 25235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:10.440653 25235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:44:10.445870 25235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:10.446949 25235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:10.447971 25235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:10.457412 25241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:10.457427 25243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:10.457971 25245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:10.458142 25235 server_base.cc:1034] running on GCE node
I20260809 04:44:10.458546 25235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:10.459154 25235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:10.460326 25235 hybrid_clock.cc:648] HybridClock initialized: now 1786250650460304 us; error 31 us; skew 500 ppm
Aug 09 04:44:10 dist-test-slave-vxj2 krb5kdc[25116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250650, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:10.463207 25235 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:44:10.464030 25235 webserver.cc:460] Webserver started at http://127.5.222.193:39721/ using document root <none> and password file <none>
I20260809 04:44:10.464600 25235 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:10.464690 25235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:10.464931 25235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:10.467024 25235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "4f2f1738476c49f8ac54d6a76c3a23cc"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-vxj2"
server_key: "0a8ed1a720b91166347cd6d752bceadd"
server_key_iv: "500ca5468a282a5a0a7c72b449c74123"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.467592 25235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "4f2f1738476c49f8ac54d6a76c3a23cc"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-vxj2"
server_key: "0a8ed1a720b91166347cd6d752bceadd"
server_key_iv: "500ca5468a282a5a0a7c72b449c74123"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.471340 25235 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:44:10.473687 25253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:10.474915 25235 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:10.475056 25235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "4f2f1738476c49f8ac54d6a76c3a23cc"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-vxj2"
server_key: "0a8ed1a720b91166347cd6d752bceadd"
server_key_iv: "500ca5468a282a5a0a7c72b449c74123"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.475167 25235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:10.492295 25235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:10.495282 25235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:10.495491 25235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:10.496106 25235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:10.496959 25235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:10.497035 25235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:10.497109 25235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:10.497151 25235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:10.506050 25235 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:41893
I20260809 04:44:10.506078 25375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:41893 every 8 connection(s)
I20260809 04:44:10.506933 25235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:10.510764 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 25235
I20260809 04:44:10.510908 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:10.511232 6011 external_mini_cluster.cc:1428] Setting key 20a4fb8d0a933b4c1e56fcfd7896c0f7
Aug 09 04:44:10 dist-test-slave-vxj2 krb5kdc[25116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250650, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
I20260809 04:44:10.522428 25210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:10.508841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:45901 (local address 127.5.222.254:41565)
0809 04:44:10.509013 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:10.509018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:10.509972 (+ 954us) server_negotiation.cc:407] Connection header received
0809 04:44:10.511665 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:10.511669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:10.511732 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:10.511829 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:10.513496 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.514017 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.515331 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.515629 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.515757 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:10.518583 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:10.518589 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:10.518593 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:10.518629 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:10.520346 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:10.520838 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:10.520844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:10.520848 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:10.520959 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:10.521356 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:10.521359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:10.521361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:10.521586 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:10.521703 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:10.522194 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 04:44:10.522299 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:44:10.523231 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:10.509235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41565 (local address 127.5.222.193:45901)
0809 04:44:10.509709 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:10.509797 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:10.511412 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:10.512141 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:10.512147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:10.512650 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:10.513335 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:10.513357 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.514144 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.514151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:10.515184 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:10.515194 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.515804 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.515808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:10.515981 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:10.516600 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:10.516615 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:10.518404 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:10.520462 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:10.520466 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:10.520476 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:10.520716 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:10.521061 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:10.521063 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:10.521065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:10.521243 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:10.521705 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:10.521712 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:10.521961 (+ 249us) client_negotiation.cc:769] Sending connection context
0809 04:44:10.522108 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 04:44:10.522347 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:44:10.524646 25376 heartbeater.cc:347] Connected to a master server at 127.5.222.254:41565
I20260809 04:44:10.524937 25376 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:10.525585 25376 heartbeater.cc:511] Master 127.5.222.254:41565 requested a full tablet report, sending...
I20260809 04:44:10.527230 25167 ts_manager.cc:194] Registered new tserver with Master: 4f2f1738476c49f8ac54d6a76c3a23cc (127.5.222.193:41893)
I20260809 04:44:10.528369 25167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:45901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:44:10.539755 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41565
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:10.652415 25381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:10.652740 25381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:10.652819 25381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:10.656831 25381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:10.656922 25381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:10.657021 25381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:44:10.662185 25381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:10.663209 25381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:10.664237 25381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:10.673804 25388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:10.673939 25381 server_base.cc:1034] running on GCE node
W20260809 04:44:10.674006 25390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:10.674118 25387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:10.674635 25381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:10.675297 25381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:10.676481 25381 hybrid_clock.cc:648] HybridClock initialized: now 1786250650676464 us; error 26 us; skew 500 ppm
Aug 09 04:44:10 dist-test-slave-vxj2 krb5kdc[25116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250650, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:10.679347 25381 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:44:10.680270 25381 webserver.cc:460] Webserver started at http://127.5.222.194:35883/ using document root <none> and password file <none>
I20260809 04:44:10.680827 25381 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:10.680920 25381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:10.681221 25381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:10.683348 25381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "03ab46013bd4441ca23ba11fb83a2006"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-vxj2"
server_key: "9b00092b00b5a28b65a1e5ab9d05ad03"
server_key_iv: "cfcbd21a91468dca4ab1a3614874ed52"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.683919 25381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "03ab46013bd4441ca23ba11fb83a2006"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-vxj2"
server_key: "9b00092b00b5a28b65a1e5ab9d05ad03"
server_key_iv: "cfcbd21a91468dca4ab1a3614874ed52"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.687621 25381 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:44:10.690289 25397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:10.691495 25381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:10.691640 25381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "03ab46013bd4441ca23ba11fb83a2006"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-vxj2"
server_key: "9b00092b00b5a28b65a1e5ab9d05ad03"
server_key_iv: "cfcbd21a91468dca4ab1a3614874ed52"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.691749 25381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:10.707249 25381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:10.710109 25381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:10.710317 25381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:10.710898 25381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:10.711720 25381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:10.711802 25381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:10.711877 25381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:10.711916 25381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:10.720997 25381 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:46461
I20260809 04:44:10.721019 25519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:46461 every 8 connection(s)
I20260809 04:44:10.721873 25381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:10.726130 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 25381
I20260809 04:44:10.726229 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:10.726580 6011 external_mini_cluster.cc:1428] Setting key b12a23012a9f88a14f8bcf81b72f8729
Aug 09 04:44:10 dist-test-slave-vxj2 krb5kdc[25116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250650, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:10.736434 25210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:10.723700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:60495 (local address 127.5.222.254:41565)
0809 04:44:10.723869 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:10.723874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:10.724662 (+ 788us) server_negotiation.cc:407] Connection header received
0809 04:44:10.725540 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:10.725543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:10.725589 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:10.725666 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:10.726911 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.727558 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.729131 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.729559 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.729669 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:10.732227 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:10.732236 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:10.732238 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:10.732260 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:10.733976 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:10.734640 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:10.734646 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:10.734649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:10.734752 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:10.735255 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:10.735259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:10.735260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:10.735454 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:10.735552 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:10.736207 (+ 655us) server_negotiation.cc:299] Negotiation successful
0809 04:44:10.736306 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 04:44:10.737629 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:10.724080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41565 (local address 127.5.222.194:60495)
0809 04:44:10.724506 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:10.724547 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:10.725347 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:10.725817 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:10.725823 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:10.726151 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:10.726768 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:10.726783 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.727688 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.727692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:10.728995 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:10.729010 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.729720 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.729727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:10.729952 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:10.730624 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:10.730640 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:10.732063 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:10.734118 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:10.734126 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:10.734140 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:10.734528 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:10.734856 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:10.734861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:10.734865 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:10.735114 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:10.735579 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:10.735586 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:10.735930 (+ 344us) client_negotiation.cc:769] Sending connection context
0809 04:44:10.736153 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 04:44:10.736495 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
I20260809 04:44:10.738850 25520 heartbeater.cc:347] Connected to a master server at 127.5.222.254:41565
I20260809 04:44:10.739102 25520 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:10.739603 25520 heartbeater.cc:511] Master 127.5.222.254:41565 requested a full tablet report, sending...
I20260809 04:44:10.740628 25167 ts_manager.cc:194] Registered new tserver with Master: 03ab46013bd4441ca23ba11fb83a2006 (127.5.222.194:46461)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
I20260809 04:44:10.741454 25167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:60495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:44:10.755710 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41565
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:10.866549 25526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:10.866799 25526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:10.866843 25526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:10.870880 25526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:10.870937 25526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:10.871034 25526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:44:10.876156 25526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:41565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:10.877185 25526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:10.878185 25526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:10.887550 25532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:10.887564 25534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:10.887641 25526 server_base.cc:1034] running on GCE node
W20260809 04:44:10.887580 25531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:10.888247 25526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:10.888887 25526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:10.890070 25526 hybrid_clock.cc:648] HybridClock initialized: now 1786250650890019 us; error 57 us; skew 500 ppm
Aug 09 04:44:10 dist-test-slave-vxj2 krb5kdc[25116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250650, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:10.893056 25526 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:44:10.893935 25526 webserver.cc:460] Webserver started at http://127.5.222.195:37535/ using document root <none> and password file <none>
I20260809 04:44:10.894515 25526 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:10.894580 25526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:10.894749 25526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:10.896816 25526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "b3c296779daa453397eec86477eddbdf"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-vxj2"
server_key: "81f2417544690295ec683d34972d11b3"
server_key_iv: "296270350ff6456d1554959f577c239a"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.897403 25526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "b3c296779daa453397eec86477eddbdf"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-vxj2"
server_key: "81f2417544690295ec683d34972d11b3"
server_key_iv: "296270350ff6456d1554959f577c239a"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.901089 25526 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:10.903679 25541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:10.904902 25526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:44:10.905041 25526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "b3c296779daa453397eec86477eddbdf"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-vxj2"
server_key: "81f2417544690295ec683d34972d11b3"
server_key_iv: "296270350ff6456d1554959f577c239a"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.905156 25526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:10.935144 25526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:10.938570 25526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:10.938793 25526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:10.939419 25526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:10.940265 25526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:10.940340 25526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:10.940412 25526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:10.940456 25526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:10.949414 25526 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:34933
I20260809 04:44:10.949461 25662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:34933 every 8 connection(s)
I20260809 04:44:10.950287 25526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:10.952602 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 25526
I20260809 04:44:10.952742 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:10.953063 6011 external_mini_cluster.cc:1428] Setting key abd86b5f6e4328bfc642171ebd073b99
Aug 09 04:44:10 dist-test-slave-vxj2 krb5kdc[25116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250650, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:10.963527 25210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:10.952032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:33085 (local address 127.5.222.254:41565)
0809 04:44:10.952177 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:10.952181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:10.953137 (+ 956us) server_negotiation.cc:407] Connection header received
0809 04:44:10.954284 (+ 1147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:10.954288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:10.954342 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:10.954430 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:10.955562 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.956112 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.957299 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.957525 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.957635 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:10.960011 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:10.960015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:10.960018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:10.960051 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:10.961541 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:10.962051 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:10.962055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:10.962056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:10.962132 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:10.962481 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:10.962485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:10.962487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:10.962661 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:10.962775 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:10.963227 (+ 452us) server_negotiation.cc:299] Negotiation successful
0809 04:44:10.963342 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:44:10.964236 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:10.952478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:41565 (local address 127.5.222.195:33085)
0809 04:44:10.952950 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:10.953015 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:10.954083 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:10.954614 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:10.954620 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:10.954957 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:10.955416 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:10.955429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.956240 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.956244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:10.957095 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:10.957104 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.957683 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.957687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:10.957859 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:10.958448 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:10.958464 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:10.959853 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:10.961659 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:10.961665 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:10.961678 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:10.961930 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:10.962224 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:10.962229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:10.962231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:10.962373 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:10.962764 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:10.962770 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:10.962992 (+ 222us) client_negotiation.cc:769] Sending connection context
0809 04:44:10.963171 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 04:44:10.963468 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":121,"threads_started":1}
I20260809 04:44:10.965407 25663 heartbeater.cc:347] Connected to a master server at 127.5.222.254:41565
I20260809 04:44:10.965674 25663 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:10.966311 25663 heartbeater.cc:511] Master 127.5.222.254:41565 requested a full tablet report, sending...
I20260809 04:44:10.967360 25167 ts_manager.cc:194] Registered new tserver with Master: b3c296779daa453397eec86477eddbdf (127.5.222.195:34933)
I20260809 04:44:10.967916 25167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:33085
I20260809 04:44:10.978739 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:44:10.988795 25210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:10.980898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.5.222.254:41565)
0809 04:44:10.981038 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:10.981042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:10.981218 (+ 176us) server_negotiation.cc:407] Connection header received
0809 04:44:10.981356 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:10.981358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:10.981401 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:10.981477 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:10.982196 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.982717 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.983721 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.983928 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.984004 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:10.985511 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:10.985515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:10.985517 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:10.985548 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:10.987005 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:10.987440 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:10.987445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:10.987447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:10.987527 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:10.987801 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:10.987805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:10.987807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:10.988133 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:10.988255 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:10.988499 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 04:44:10.988615 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:44:10.991766 25167 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:10.994875 25167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:11.009364 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.003057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:34933 (local address 127.0.0.1:55652)
0809 04:44:11.004204 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:11.004271 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:11.004490 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:11.005107 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:11.005113 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:11.005135 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:11.005320 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.005327 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.006426 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.006439 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.007670 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.007701 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.008751 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.008755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.008889 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.008926 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 04:44:11.009050 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 04:44:11.009198 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":71,"threads_started":1}
I20260809 04:44:11.009392 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.003267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55652 (local address 127.5.222.195:34933)
0809 04:44:11.003662 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:11.003675 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:11.004373 (+ 698us) server_negotiation.cc:407] Connection header received
0809 04:44:11.004695 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:11.004703 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:11.004930 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:11.005093 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:11.005451 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.006289 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.007800 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.008368 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.008500 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.008654 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:11.009099 (+ 445us) server_negotiation.cc:299] Negotiation successful
0809 04:44:11.009239 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":202,"threads_started":1}
I20260809 04:44:11.010267 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.003250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:46461 (local address 127.0.0.1:50776)
0809 04:44:11.004451 (+ 1201us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:11.004466 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:11.004619 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:11.005221 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:11.005224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:11.005237 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:11.005320 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.005326 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.006989 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.006992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.008397 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.008407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.009719 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.009722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.009902 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.009927 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:44:11.010027 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 04:44:11.010100 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1132,"mutex_wait_us":68,"thread_start_us":95,"threads_started":1}
I20260809 04:44:11.010691 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.003858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50776 (local address 127.5.222.194:46461)
0809 04:44:11.004232 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:11.004245 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:11.004567 (+ 322us) server_negotiation.cc:407] Connection header received
0809 04:44:11.004768 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:11.004776 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:11.004983 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:11.005143 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:11.005587 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.006821 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.008567 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.009401 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.009514 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.009650 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:11.010458 (+ 808us) server_negotiation.cc:299] Negotiation successful
0809 04:44:11.010561 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":91,"threads_started":1}
I20260809 04:44:11.010902 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.004029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.5.222.193:41893)
0809 04:44:11.004451 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:11.004468 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:11.004862 (+ 394us) server_negotiation.cc:407] Connection header received
0809 04:44:11.004939 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:11.004944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:11.005111 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:11.005282 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:11.005711 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.006835 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.008492 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.009383 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.009515 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.009663 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:11.010621 (+ 958us) server_negotiation.cc:299] Negotiation successful
0809 04:44:11.010728 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":134,"threads_started":1}
I20260809 04:44:11.010519 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.003810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:41893 (local address 127.0.0.1:38736)
0809 04:44:11.004742 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:11.004755 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:11.004852 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:11.005464 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:11.005467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:11.005486 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:11.005579 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.005584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.006989 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.006992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.008340 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.008352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.009725 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.009729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.009865 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.010192 (+ 327us) client_negotiation.cc:769] Sending connection context
0809 04:44:11.010264 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 04:44:11.010342 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"mutex_wait_us":498,"thread_start_us":104,"threads_started":1}
I20260809 04:44:11.013118 25591 tablet_service.cc:1467] Processing CreateTablet for tablet a5ecac2c3f6a4cc2ab57409566a7a739 (DEFAULT_TABLE table=test-table [id=7677d296a9dc48858c336eb7ee248a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:11.013455 25447 tablet_service.cc:1467] Processing CreateTablet for tablet a5ecac2c3f6a4cc2ab57409566a7a739 (DEFAULT_TABLE table=test-table [id=7677d296a9dc48858c336eb7ee248a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:11.014355 25591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5ecac2c3f6a4cc2ab57409566a7a739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:11.014478 25447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5ecac2c3f6a4cc2ab57409566a7a739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:11.014225 25305 tablet_service.cc:1467] Processing CreateTablet for tablet a5ecac2c3f6a4cc2ab57409566a7a739 (DEFAULT_TABLE table=test-table [id=7677d296a9dc48858c336eb7ee248a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:11.015157 25305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5ecac2c3f6a4cc2ab57409566a7a739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:11.020689 25680 tablet_bootstrap.cc:492] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf: Bootstrap starting.
I20260809 04:44:11.022549 25681 tablet_bootstrap.cc:492] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006: Bootstrap starting.
I20260809 04:44:11.023072 25680 tablet_bootstrap.cc:654] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:11.024170 25680 log.cc:824] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:11.024871 25681 tablet_bootstrap.cc:654] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:11.024967 25682 tablet_bootstrap.cc:492] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc: Bootstrap starting.
I20260809 04:44:11.025655 25681 log.cc:824] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:11.026760 25682 tablet_bootstrap.cc:654] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:11.027410 25682 log.cc:824] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:11.027513 25681 tablet_bootstrap.cc:492] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006: No bootstrap required, opened a new log
I20260809 04:44:11.027716 25681 ts_tablet_manager.cc:1397] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:44:11.029402 25680 tablet_bootstrap.cc:492] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf: No bootstrap required, opened a new log
I20260809 04:44:11.029397 25682 tablet_bootstrap.cc:492] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc: No bootstrap required, opened a new log
I20260809 04:44:11.029593 25680 ts_tablet_manager.cc:1397] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260809 04:44:11.029593 25682 ts_tablet_manager.cc:1397] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:44:11.031023 25681 raft_consensus.cc:348] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c296779daa453397eec86477eddbdf" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34933 } } peers { permanent_uuid: "03ab46013bd4441ca23ba11fb83a2006" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46461 } } peers { permanent_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41893 } }
I20260809 04:44:11.031283 25681 raft_consensus.cc:374] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:11.031356 25681 raft_consensus.cc:729] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ab46013bd4441ca23ba11fb83a2006, State: Initialized, Role: FOLLOWER
I20260809 04:44:11.031832 25681 consensus_queue.cc:260] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c296779daa453397eec86477eddbdf" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34933 } } peers { permanent_uuid: "03ab46013bd4441ca23ba11fb83a2006" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46461 } } peers { permanent_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41893 } }
I20260809 04:44:11.032469 25520 heartbeater.cc:503] Master 127.5.222.254:41565 was elected leader, sending a full tablet report...
I20260809 04:44:11.033771 25681 ts_tablet_manager.cc:1428] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:44:11.033859 25682 raft_consensus.cc:348] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c296779daa453397eec86477eddbdf" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34933 } } peers { permanent_uuid: "03ab46013bd4441ca23ba11fb83a2006" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46461 } } peers { permanent_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41893 } }
I20260809 04:44:11.034155 25682 raft_consensus.cc:374] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:11.034219 25682 raft_consensus.cc:729] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f2f1738476c49f8ac54d6a76c3a23cc, State: Initialized, Role: FOLLOWER
I20260809 04:44:11.034649 25682 consensus_queue.cc:260] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c296779daa453397eec86477eddbdf" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34933 } } peers { permanent_uuid: "03ab46013bd4441ca23ba11fb83a2006" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46461 } } peers { permanent_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41893 } }
I20260809 04:44:11.034765 25680 raft_consensus.cc:348] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c296779daa453397eec86477eddbdf" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34933 } } peers { permanent_uuid: "03ab46013bd4441ca23ba11fb83a2006" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46461 } } peers { permanent_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41893 } }
I20260809 04:44:11.035130 25680 raft_consensus.cc:374] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:11.035228 25680 raft_consensus.cc:729] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3c296779daa453397eec86477eddbdf, State: Initialized, Role: FOLLOWER
I20260809 04:44:11.035341 25376 heartbeater.cc:503] Master 127.5.222.254:41565 was elected leader, sending a full tablet report...
I20260809 04:44:11.035682 25682 ts_tablet_manager.cc:1428] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:44:11.035724 25680 consensus_queue.cc:260] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c296779daa453397eec86477eddbdf" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34933 } } peers { permanent_uuid: "03ab46013bd4441ca23ba11fb83a2006" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46461 } } peers { permanent_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41893 } }
I20260809 04:44:11.036422 25663 heartbeater.cc:503] Master 127.5.222.254:41565 was elected leader, sending a full tablet report...
I20260809 04:44:11.036860 25680 ts_tablet_manager.cc:1428] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 04:44:11.054147 25686 raft_consensus.cc:482] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:11.054365 25686 raft_consensus.cc:504] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c296779daa453397eec86477eddbdf" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34933 } } peers { permanent_uuid: "03ab46013bd4441ca23ba11fb83a2006" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46461 } } peers { permanent_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41893 } }
I20260809 04:44:11.055461 25686 leader_election.cc:288] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3c296779daa453397eec86477eddbdf (127.5.222.195:34933), 4f2f1738476c49f8ac54d6a76c3a23cc (127.5.222.193:41893)
I20260809 04:44:11.059578 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.055684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:34933 (local address 127.5.222.194:46791)
0809 04:44:11.055802 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:11.055820 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:11.055910 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:11.056247 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:11.056250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:11.056270 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:11.056404 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.056414 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.057349 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.057352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.058497 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.058508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.059241 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.059244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.059337 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.059357 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:44:11.059421 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 04:44:11.059476 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":3}
I20260809 04:44:11.059669 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.055833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:46791 (local address 127.5.222.195:34933)
0809 04:44:11.055999 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:11.056003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:11.056018 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:44:11.056067 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:11.056073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:11.056129 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:11.056211 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:11.056529 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.057238 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.058623 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.059108 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.059226 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.059275 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:11.059463 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 04:44:11.059534 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:44:11.060222 25689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.055684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:41893 (local address 127.5.222.194:51375)
0809 04:44:11.056337 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:11.056352 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:11.056454 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:11.056718 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:11.056721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:11.056739 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:11.056884 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.056889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.057749 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.057752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.058865 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.058874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.059843 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.059849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.059943 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.059967 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:44:11.060049 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 04:44:11.060116 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":320,"thread_start_us":137,"threads_started":1}
I20260809 04:44:11.060286 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.055823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:51375 (local address 127.5.222.193:41893)
0809 04:44:11.055968 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:11.055972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:11.056448 (+ 476us) server_negotiation.cc:407] Connection header received
0809 04:44:11.056540 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:11.056543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:11.056592 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:11.056674 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:11.057001 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.057632 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.058987 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.059581 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.059700 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.059755 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:11.060087 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 04:44:11.060167 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:44:11.060355 25614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ecac2c3f6a4cc2ab57409566a7a739" candidate_uuid: "03ab46013bd4441ca23ba11fb83a2006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3c296779daa453397eec86477eddbdf" is_pre_election: true
I20260809 04:44:11.060658 25614 raft_consensus.cc:2454] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03ab46013bd4441ca23ba11fb83a2006 in term 0.
I20260809 04:44:11.061024 25327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ecac2c3f6a4cc2ab57409566a7a739" candidate_uuid: "03ab46013bd4441ca23ba11fb83a2006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc" is_pre_election: true
I20260809 04:44:11.061203 25399 leader_election.cc:302] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03ab46013bd4441ca23ba11fb83a2006, b3c296779daa453397eec86477eddbdf; no voters:
I20260809 04:44:11.061338 25327 raft_consensus.cc:2454] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03ab46013bd4441ca23ba11fb83a2006 in term 0.
I20260809 04:44:11.061493 25686 raft_consensus.cc:2781] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:11.061610 25686 raft_consensus.cc:482] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:11.061668 25686 raft_consensus.cc:3037] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:11.062769 25686 raft_consensus.cc:504] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c296779daa453397eec86477eddbdf" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34933 } } peers { permanent_uuid: "03ab46013bd4441ca23ba11fb83a2006" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46461 } } peers { permanent_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41893 } }
I20260809 04:44:11.063169 25686 leader_election.cc:288] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [CANDIDATE]: Term 1 election: Requested vote from peers b3c296779daa453397eec86477eddbdf (127.5.222.195:34933), 4f2f1738476c49f8ac54d6a76c3a23cc (127.5.222.193:41893)
I20260809 04:44:11.063570 25614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ecac2c3f6a4cc2ab57409566a7a739" candidate_uuid: "03ab46013bd4441ca23ba11fb83a2006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3c296779daa453397eec86477eddbdf"
I20260809 04:44:11.063642 25327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ecac2c3f6a4cc2ab57409566a7a739" candidate_uuid: "03ab46013bd4441ca23ba11fb83a2006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc"
I20260809 04:44:11.063725 25614 raft_consensus.cc:3037] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:11.063748 25327 raft_consensus.cc:3037] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:11.065011 25614 raft_consensus.cc:2454] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03ab46013bd4441ca23ba11fb83a2006 in term 1.
I20260809 04:44:11.065009 25327 raft_consensus.cc:2454] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03ab46013bd4441ca23ba11fb83a2006 in term 1.
I20260809 04:44:11.065358 25398 leader_election.cc:302] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03ab46013bd4441ca23ba11fb83a2006, 4f2f1738476c49f8ac54d6a76c3a23cc; no voters:
I20260809 04:44:11.065551 25686 raft_consensus.cc:2781] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:11.065795 25686 raft_consensus.cc:686] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 1 LEADER]: Becoming Leader. State: Replica: 03ab46013bd4441ca23ba11fb83a2006, State: Running, Role: LEADER
I20260809 04:44:11.066111 25686 consensus_queue.cc:237] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c296779daa453397eec86477eddbdf" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34933 } } peers { permanent_uuid: "03ab46013bd4441ca23ba11fb83a2006" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46461 } } peers { permanent_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41893 } }
I20260809 04:44:11.069131 25166 catalog_manager.cc:5495] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03ab46013bd4441ca23ba11fb83a2006 (127.5.222.194). New cstate: current_term: 1 leader_uuid: "03ab46013bd4441ca23ba11fb83a2006" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c296779daa453397eec86477eddbdf" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03ab46013bd4441ca23ba11fb83a2006" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 46461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41893 } health_report { overall_health: UNKNOWN } } }
I20260809 04:44:11.087327 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.083865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.5.222.194:46461)
0809 04:44:11.084010 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:11.084015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:11.084029 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:44:11.084125 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:11.084127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:11.084176 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:11.084286 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:11.084526 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.085020 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.085914 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.086133 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.086237 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.086559 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:11.086931 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:11.087070 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:11.087131 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 04:44:11.087186 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:44:11.093283 25327 raft_consensus.cc:1261] T a5ecac2c3f6a4cc2ab57409566a7a739 P 4f2f1738476c49f8ac54d6a76c3a23cc [term 1 FOLLOWER]: Refusing update from remote peer 03ab46013bd4441ca23ba11fb83a2006: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:11.093283 25614 raft_consensus.cc:1261] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf [term 1 FOLLOWER]: Refusing update from remote peer 03ab46013bd4441ca23ba11fb83a2006: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:11.094062 25686 consensus_queue.cc:1035] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3c296779daa453397eec86477eddbdf" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:11.094306 25690 consensus_queue.cc:1035] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f2f1738476c49f8ac54d6a76c3a23cc" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 41893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:11.103046 25694 mvcc.cc:205] Tried to move back new op lower bound from 7316482666870087680 to 7316482666769223680. Current Snapshot: MvccSnapshot[applied={T|T < 7316482666870087680}]
I20260809 04:44:11.104995 25695 mvcc.cc:205] Tried to move back new op lower bound from 7316482666870087680 to 7316482666769223680. Current Snapshot: MvccSnapshot[applied={T|T < 7316482666870087680}]
I20260809 04:44:11.111418 25167 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:11.111617 25167 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:11.114454 25167 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25842abfa1ea4de898286fd4adf622f1: Sending DeleteTablet for 3 replicas of tablet a5ecac2c3f6a4cc2ab57409566a7a739
I20260809 04:44:11.115581 25591 tablet_service.cc:1514] Processing DeleteTablet for tablet a5ecac2c3f6a4cc2ab57409566a7a739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:11 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:55652
I20260809 04:44:11.115670 25305 tablet_service.cc:1514] Processing DeleteTablet for tablet a5ecac2c3f6a4cc2ab57409566a7a739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:11 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:38736
I20260809 04:44:11.116148 25703 tablet_replica.cc:330] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf: stopping tablet replica
I20260809 04:44:11.116243 25447 tablet_service.cc:1514] Processing DeleteTablet for tablet a5ecac2c3f6a4cc2ab57409566a7a739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:11 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:50776
I20260809 04:44:11.116508 25703 raft_consensus.cc:2229] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:11.116837 25703 raft_consensus.cc:2258] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:11.116858 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 25235
I20260809 04:44:11.116869 25705 tablet_replica.cc:330] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006: stopping tablet replica
I20260809 04:44:11.117144 25705 raft_consensus.cc:2229] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:11.117712 25705 raft_consensus.cc:2258] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:11.119303 25703 ts_tablet_manager.cc:1905] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:11.119350 25705 ts_tablet_manager.cc:1905] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:11.122239 25705 ts_tablet_manager.cc:1918] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:11.122411 25705 log.cc:1196] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/a5ecac2c3f6a4cc2ab57409566a7a739
I20260809 04:44:11.122781 25703 ts_tablet_manager.cc:1918] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:11.122828 25705 ts_tablet_manager.cc:1939] T a5ecac2c3f6a4cc2ab57409566a7a739 P 03ab46013bd4441ca23ba11fb83a2006: Deleting consensus metadata
I20260809 04:44:11.122991 25703 log.cc:1196] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/a5ecac2c3f6a4cc2ab57409566a7a739
I20260809 04:44:11.123458 25703 ts_tablet_manager.cc:1939] T a5ecac2c3f6a4cc2ab57409566a7a739 P b3c296779daa453397eec86477eddbdf: Deleting consensus metadata
I20260809 04:44:11.124038 25152 catalog_manager.cc:4841] TS 03ab46013bd4441ca23ba11fb83a2006 (127.5.222.194:46461): tablet a5ecac2c3f6a4cc2ab57409566a7a739 (table test-table [id=7677d296a9dc48858c336eb7ee248a91]) successfully deleted
I20260809 04:44:11.124671 25149 catalog_manager.cc:4841] TS b3c296779daa453397eec86477eddbdf (127.5.222.195:34933): tablet a5ecac2c3f6a4cc2ab57409566a7a739 (table test-table [id=7677d296a9dc48858c336eb7ee248a91]) successfully deleted
W20260809 04:44:11.128634 25148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:41893 (error 108)
I20260809 04:44:11.129534 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 25381
I20260809 04:44:11.129748 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.129434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:41893 (local address 127.0.0.1:38738)
0809 04:44:11.129566 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:11.129648 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:41893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 04:44:11.130734 25148 catalog_manager.cc:4584] TS 4f2f1738476c49f8ac54d6a76c3a23cc (127.5.222.193:41893): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5ecac2c3f6a4cc2ab57409566a7a739: Network error: Client connection negotiation failed: client connection to 127.5.222.193:41893: connect: Connection refused (error 111)
I20260809 04:44:11.137704 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 25526
I20260809 04:44:11.144390 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 25129
2026-08-09T04:44:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2339 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T04:44:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:11Z Disabled control of system clock
I20260809 04:44:11.176698 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:46869
--webserver_interface=127.5.222.254
--webserver_port=0
--builtin_ntp_servers=127.5.222.212:38073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:46869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:44:11.305721 25710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:11.305977 25710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:11.306013 25710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:11.309894 25710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:11.309947 25710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:11.309962 25710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:11.309978 25710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:11.309994 25710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:11.314680 25710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:38073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:46869
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:46869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:11.315774 25710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:11.317126 25710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:11.325842 25716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:11.325852 25720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:11.326165 25710 server_base.cc:1034] running on GCE node
W20260809 04:44:11.325852 25717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:11.326951 25710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:11.328200 25710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:11.329376 25710 hybrid_clock.cc:648] HybridClock initialized: now 1786250651329356 us; error 35 us; skew 500 ppm
I20260809 04:44:11.331184 25710 webserver.cc:460] Webserver started at http://127.5.222.254:38577/ using document root <none> and password file <none>
I20260809 04:44:11.331813 25710 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:11.331875 25710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:11.332118 25710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:11.334229 25710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "0fe6d761665645ee9168cc8ebec59a26"
format_stamp: "Formatted at 2026-08-09 04:44:11 on dist-test-slave-vxj2"
server_key: "1bc26832e4c560d077ebf402fc2cd576"
server_key_iv: "e9e3ae0586c2739e8d4a7f1c73f4cadc"
server_key_version: "encryptionkey@0"
I20260809 04:44:11.334823 25710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "0fe6d761665645ee9168cc8ebec59a26"
format_stamp: "Formatted at 2026-08-09 04:44:11 on dist-test-slave-vxj2"
server_key: "1bc26832e4c560d077ebf402fc2cd576"
server_key_iv: "e9e3ae0586c2739e8d4a7f1c73f4cadc"
server_key_version: "encryptionkey@0"
I20260809 04:44:11.338389 25710 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:44:11.340787 25727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:11.342007 25710 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:11.342141 25710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "0fe6d761665645ee9168cc8ebec59a26"
format_stamp: "Formatted at 2026-08-09 04:44:11 on dist-test-slave-vxj2"
server_key: "1bc26832e4c560d077ebf402fc2cd576"
server_key_iv: "e9e3ae0586c2739e8d4a7f1c73f4cadc"
server_key_version: "encryptionkey@0"
I20260809 04:44:11.342253 25710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:11.357489 25710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:11.358142 25710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:11.358337 25710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:11.365787 25710 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:46869
I20260809 04:44:11.365849 25784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:46869 every 8 connection(s)
I20260809 04:44:11.366780 25710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:11.369997 25785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:11.373624 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 25710
I20260809 04:44:11.373760 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:11.374092 6011 external_mini_cluster.cc:1428] Setting key 31e84218ceef4afa5dc1de28d606ff5c
I20260809 04:44:11.376091 25785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26: Bootstrap starting.
I20260809 04:44:11.378299 25785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:11.379026 25785 log.cc:824] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:11.381174 25785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26: No bootstrap required, opened a new log
I20260809 04:44:11.386134 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.375617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58408 (local address 127.5.222.254:46869)
0809 04:44:11.376298 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:11.376314 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:11.376349 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:44:11.376979 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:11.376995 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:11.377452 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:11.377778 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:11.378217 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.379271 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.380278 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.381100 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.382088 (+ 988us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.383390 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:11.383400 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:11.383426 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:11.383999 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:11.384080 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:11.384112 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:11.384219 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:11.384805 (+ 586us) server_negotiation.cc:299] Negotiation successful
0809 04:44:11.385101 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":122,"threads_started":1}
I20260809 04:44:11.386762 25785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe6d761665645ee9168cc8ebec59a26" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 46869 } }
I20260809 04:44:11.387025 25785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:11.387123 25785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe6d761665645ee9168cc8ebec59a26, State: Initialized, Role: FOLLOWER
I20260809 04:44:11.387583 25785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe6d761665645ee9168cc8ebec59a26" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 46869 } }
I20260809 04:44:11.387773 25785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:11.387854 25785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:11.387967 25785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:11.389186 25785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe6d761665645ee9168cc8ebec59a26" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 46869 } }
I20260809 04:44:11.389533 25785 leader_election.cc:302] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fe6d761665645ee9168cc8ebec59a26; no voters:
I20260809 04:44:11.389909 25785 leader_election.cc:288] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:11.390012 25790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:11.390275 25790 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [term 1 LEADER]: Becoming Leader. State: Replica: 0fe6d761665645ee9168cc8ebec59a26, State: Running, Role: LEADER
I20260809 04:44:11.390606 25790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe6d761665645ee9168cc8ebec59a26" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 46869 } }
I20260809 04:44:11.391255 25785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:11.392464 25792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fe6d761665645ee9168cc8ebec59a26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe6d761665645ee9168cc8ebec59a26" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 46869 } } }
I20260809 04:44:11.392604 25792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:11.393029 25800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:11.392978 25793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fe6d761665645ee9168cc8ebec59a26. Latest consensus state: current_term: 1 leader_uuid: "0fe6d761665645ee9168cc8ebec59a26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe6d761665645ee9168cc8ebec59a26" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 46869 } } }
I20260809 04:44:11.393131 25793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:11.396265 25800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:11.402331 25800 catalog_manager.cc:1347] Generated new cluster ID: d53905c9e1ba4e0fa9d951e9d90b9999
I20260809 04:44:11.402397 25800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:11.441677 25800 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:11.442986 25800 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:11.463732 25800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26: Generated new TSK 0
I20260809 04:44:11.464630 25800 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:11.473114 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:46869
--builtin_ntp_servers=127.5.222.212:38073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:44:11.586083 25810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:11.586352 25810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:11.586390 25810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:11.590723 25810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:11.590788 25810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:11.590921 25810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:44:11.595978 25810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:38073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:46869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:11.597119 25810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:11.598635 25810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:11.608655 25816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:11.608655 25818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:11.608704 25810 server_base.cc:1034] running on GCE node
W20260809 04:44:11.608662 25815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:11.609380 25810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:11.610206 25810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:11.611385 25810 hybrid_clock.cc:648] HybridClock initialized: now 1786250651611368 us; error 31 us; skew 500 ppm
I20260809 04:44:11.616405 25810 webserver.cc:460] Webserver started at http://127.5.222.193:41137/ using document root <none> and password file <none>
I20260809 04:44:11.617237 25810 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:11.617367 25810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:11.617642 25810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:11.620050 25810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "3912dec245944d4cb2156381bde24162"
format_stamp: "Formatted at 2026-08-09 04:44:11 on dist-test-slave-vxj2"
server_key: "d1d2f73d968910787a6b63b32bb55734"
server_key_iv: "f5844fba9e17562801b2ac6b96d7bb46"
server_key_version: "encryptionkey@0"
I20260809 04:44:11.620586 25810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "3912dec245944d4cb2156381bde24162"
format_stamp: "Formatted at 2026-08-09 04:44:11 on dist-test-slave-vxj2"
server_key: "d1d2f73d968910787a6b63b32bb55734"
server_key_iv: "f5844fba9e17562801b2ac6b96d7bb46"
server_key_version: "encryptionkey@0"
I20260809 04:44:11.624229 25810 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 04:44:11.626642 25824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:11.627909 25810 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 04:44:11.628036 25810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "3912dec245944d4cb2156381bde24162"
format_stamp: "Formatted at 2026-08-09 04:44:11 on dist-test-slave-vxj2"
server_key: "d1d2f73d968910787a6b63b32bb55734"
server_key_iv: "f5844fba9e17562801b2ac6b96d7bb46"
server_key_version: "encryptionkey@0"
I20260809 04:44:11.628155 25810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:11.655442 25810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:11.656136 25810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:11.656335 25810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:11.656926 25810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:11.657889 25810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:11.657939 25810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:11.658015 25810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:11.658046 25810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:11.667449 25810 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:38135
I20260809 04:44:11.667470 25943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:38135 every 8 connection(s)
I20260809 04:44:11.668313 25810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:11.670352 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 25810
I20260809 04:44:11.670490 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:11.670773 6011 external_mini_cluster.cc:1428] Setting key fbf8dd17bca33a5250414999019f7d1e
I20260809 04:44:11.672695 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:46869
--builtin_ntp_servers=127.5.222.212:38073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:44:11.678623 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.670194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:37561 (local address 127.5.222.254:46869)
0809 04:44:11.670367 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:11.670374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:11.671484 (+ 1110us) server_negotiation.cc:407] Connection header received
0809 04:44:11.673038 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:11.673045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:11.673108 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:11.673235 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:11.674284 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.674822 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.676164 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.676571 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.676708 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.677931 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:11.677945 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:11.677951 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:11.677970 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:11.677997 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:11.678012 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:11.678110 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:11.678323 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 04:44:11.678406 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:44:11.679455 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.670630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:46869 (local address 127.5.222.193:37561)
0809 04:44:11.671329 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:11.671380 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:11.672801 (+ 1421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:11.673406 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:11.673413 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:11.673509 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:11.674107 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.674130 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.674953 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.674956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.676023 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.676033 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.676732 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.676735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.676985 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.677661 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:11.677662 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:11.677701 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:11.677769 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:11.678148 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:11.678157 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:11.678163 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:44:11.678293 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 04:44:11.678536 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":143,"threads_started":1}
I20260809 04:44:11.680847 25944 heartbeater.cc:347] Connected to a master server at 127.5.222.254:46869
I20260809 04:44:11.681088 25944 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:11.681686 25944 heartbeater.cc:511] Master 127.5.222.254:46869 requested a full tablet report, sending...
I20260809 04:44:11.683283 25745 ts_manager.cc:194] Registered new tserver with Master: 3912dec245944d4cb2156381bde24162 (127.5.222.193:38135)
I20260809 04:44:11.684530 25745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.222.193:37561
W20260809 04:44:11.799693 25947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:11.799934 25947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:11.799974 25947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:11.803974 25947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:11.804029 25947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:11.804114 25947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:44:11.808820 25947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:38073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:46869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:11.809931 25947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:11.811336 25947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:11.820989 25952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:11.821024 25953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:11.821177 25947 server_base.cc:1034] running on GCE node
W20260809 04:44:11.821372 25955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:11.821743 25947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:11.822417 25947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:11.823617 25947 hybrid_clock.cc:648] HybridClock initialized: now 1786250651823601 us; error 27 us; skew 500 ppm
I20260809 04:44:11.825474 25947 webserver.cc:460] Webserver started at http://127.5.222.194:43197/ using document root <none> and password file <none>
I20260809 04:44:11.826052 25947 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:11.826143 25947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:11.826382 25947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:11.828418 25947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "dd76778620cd4599adfb01859440c3aa"
format_stamp: "Formatted at 2026-08-09 04:44:11 on dist-test-slave-vxj2"
server_key: "14578e9513d7c49a7cb8d823bfb55013"
server_key_iv: "2a7712e74c2d0b75ec2e8a5c41d02fbd"
server_key_version: "encryptionkey@0"
I20260809 04:44:11.828984 25947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "dd76778620cd4599adfb01859440c3aa"
format_stamp: "Formatted at 2026-08-09 04:44:11 on dist-test-slave-vxj2"
server_key: "14578e9513d7c49a7cb8d823bfb55013"
server_key_iv: "2a7712e74c2d0b75ec2e8a5c41d02fbd"
server_key_version: "encryptionkey@0"
I20260809 04:44:11.832727 25947 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:11.835158 25961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:11.836361 25947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:44:11.836491 25947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "dd76778620cd4599adfb01859440c3aa"
format_stamp: "Formatted at 2026-08-09 04:44:11 on dist-test-slave-vxj2"
server_key: "14578e9513d7c49a7cb8d823bfb55013"
server_key_iv: "2a7712e74c2d0b75ec2e8a5c41d02fbd"
server_key_version: "encryptionkey@0"
I20260809 04:44:11.836601 25947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:11.859624 25947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:11.860396 25947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:11.860601 25947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:11.861268 25947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:11.862201 25947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:11.862280 25947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:11.862351 25947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:11.862401 25947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:11.871349 25947 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:37079
I20260809 04:44:11.871423 26082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:37079 every 8 connection(s)
I20260809 04:44:11.872249 25947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:11.880298 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 25947
I20260809 04:44:11.880456 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:11.880759 6011 external_mini_cluster.cc:1428] Setting key 3e7da4bf39fdeeb05692f209959f7a39
I20260809 04:44:11.881664 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.874145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:44471 (local address 127.5.222.254:46869)
0809 04:44:11.874345 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:11.874349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:11.875341 (+ 992us) server_negotiation.cc:407] Connection header received
0809 04:44:11.876678 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:11.876680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:11.876727 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:11.876804 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:11.877774 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.878349 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.879456 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.879665 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.879761 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.881068 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:11.881072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:11.881075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:11.881093 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:11.881111 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:11.881118 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:11.881266 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:11.881442 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 04:44:11.881523 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 04:44:11.882467 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.874569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:46869 (local address 127.5.222.194:44471)
0809 04:44:11.875199 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:11.875238 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:11.876477 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:11.876977 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:11.876983 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:11.877060 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:11.877618 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.877630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.878478 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.878480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.879337 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.879345 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.879799 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.879802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.879933 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.880755 (+ 822us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:11.880757 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:11.880827 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:11.880913 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:11.881268 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:11.881280 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:11.881288 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:44:11.881418 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 04:44:11.881665 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":110,"threads_started":1}
I20260809 04:44:11.882804 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:46869
--builtin_ntp_servers=127.5.222.212:38073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:44:11.883595 26083 heartbeater.cc:347] Connected to a master server at 127.5.222.254:46869
I20260809 04:44:11.883836 26083 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:11.884455 26083 heartbeater.cc:511] Master 127.5.222.254:46869 requested a full tablet report, sending...
I20260809 04:44:11.885668 25745 ts_manager.cc:194] Registered new tserver with Master: dd76778620cd4599adfb01859440c3aa (127.5.222.194:37079)
I20260809 04:44:11.886541 25745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.222.194:44471
W20260809 04:44:12.002955 26086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:12.003237 26086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:12.003278 26086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:12.007583 26086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:12.007643 26086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:12.007699 26086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:44:12.012531 26086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:38073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:46869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:12.013686 26086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:12.015072 26086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:12.024737 26092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:12.024745 26091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:12.024751 26094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:12.024889 26086 server_base.cc:1034] running on GCE node
I20260809 04:44:12.025440 26086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:12.026121 26086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:12.027349 26086 hybrid_clock.cc:648] HybridClock initialized: now 1786250652027315 us; error 50 us; skew 500 ppm
I20260809 04:44:12.029439 26086 webserver.cc:460] Webserver started at http://127.5.222.195:33853/ using document root <none> and password file <none>
I20260809 04:44:12.030037 26086 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:12.030162 26086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:12.030419 26086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:12.032645 26086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "5f43ec14442f4e8cadafba82dc7feca3"
format_stamp: "Formatted at 2026-08-09 04:44:12 on dist-test-slave-vxj2"
server_key: "e7b83e0a0562189b542b44d1c660ac44"
server_key_iv: "8fb9533d32b946273e27f431dcbb2234"
server_key_version: "encryptionkey@0"
I20260809 04:44:12.033326 26086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "5f43ec14442f4e8cadafba82dc7feca3"
format_stamp: "Formatted at 2026-08-09 04:44:12 on dist-test-slave-vxj2"
server_key: "e7b83e0a0562189b542b44d1c660ac44"
server_key_iv: "8fb9533d32b946273e27f431dcbb2234"
server_key_version: "encryptionkey@0"
I20260809 04:44:12.037214 26086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:44:12.039731 26100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:12.041050 26086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:44:12.041204 26086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "5f43ec14442f4e8cadafba82dc7feca3"
format_stamp: "Formatted at 2026-08-09 04:44:12 on dist-test-slave-vxj2"
server_key: "e7b83e0a0562189b542b44d1c660ac44"
server_key_iv: "8fb9533d32b946273e27f431dcbb2234"
server_key_version: "encryptionkey@0"
I20260809 04:44:12.041332 26086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:12.063473 26086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:12.064199 26086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:12.064417 26086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:12.065064 26086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:12.066095 26086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:12.066169 26086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:12.066243 26086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:12.066298 26086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:12.075204 26086 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:38507
I20260809 04:44:12.075234 26223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:38507 every 8 connection(s)
I20260809 04:44:12.076082 26086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:12.080561 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 26086
I20260809 04:44:12.080679 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:12.081043 6011 external_mini_cluster.cc:1428] Setting key cd9214202f4832b17e016efbec4a866e
I20260809 04:44:12.085727 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.078001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:34073 (local address 127.5.222.254:46869)
0809 04:44:12.078153 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:12.078156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:12.079173 (+ 1017us) server_negotiation.cc:407] Connection header received
0809 04:44:12.080825 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:12.080828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:12.080880 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:12.080965 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:12.082038 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.082646 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.083785 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.084015 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.084124 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.085241 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:12.085245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:12.085248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:12.085260 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:12.085287 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:12.085295 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:12.085375 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:12.085527 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 04:44:12.085585 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:44:12.086552 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.078378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:46869 (local address 127.5.222.195:34073)
0809 04:44:12.079025 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:12.079065 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:12.080525 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:12.081181 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:12.081199 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:12.081302 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:12.081868 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:12.081889 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.082780 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.082783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:12.083650 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:12.083659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.084162 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.084166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:12.084325 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.084955 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:12.084957 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:12.084995 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:12.085055 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:12.085376 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:12.085387 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:12.085395 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:44:12.085527 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 04:44:12.085806 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":117,"threads_started":1}
I20260809 04:44:12.087568 26224 heartbeater.cc:347] Connected to a master server at 127.5.222.254:46869
I20260809 04:44:12.087767 26224 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:12.088209 26224 heartbeater.cc:511] Master 127.5.222.254:46869 requested a full tablet report, sending...
I20260809 04:44:12.089191 25745 ts_manager.cc:194] Registered new tserver with Master: 5f43ec14442f4e8cadafba82dc7feca3 (127.5.222.195:38507)
I20260809 04:44:12.089716 25745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.222.195:34073
I20260809 04:44:12.094728 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:44:12.100507 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.096716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58416 (local address 127.5.222.254:46869)
0809 04:44:12.096950 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:12.096953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:12.097018 (+ 65us) server_negotiation.cc:407] Connection header received
0809 04:44:12.097132 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:12.097134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:12.097218 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:12.097290 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:12.097604 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.098126 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.099045 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.099243 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.099315 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.100055 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:12.100058 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:12.100060 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:12.100071 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:12.100088 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:12.100095 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:12.100153 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:12.100265 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 04:44:12.100320 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260809 04:44:12.103391 25745 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:12.106599 25745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:12.121129 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.115140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:37079 (local address 127.0.0.1:50086)
0809 04:44:12.116229 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:12.116243 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:12.116344 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:12.117121 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:12.117123 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:12.117141 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:12.117252 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:12.117258 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.118698 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.118701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:12.119833 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:12.119843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.120627 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.120630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:12.120710 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.120740 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 04:44:12.120786 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 04:44:12.120967 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"mutex_wait_us":67,"thread_start_us":89,"threads_started":1}
I20260809 04:44:12.121459 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.115859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45686 (local address 127.5.222.195:38507)
0809 04:44:12.116267 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:12.116283 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:12.116308 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:44:12.116389 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:12.116396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:12.116628 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:12.116788 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:12.117122 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.118345 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.119914 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.120674 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.120780 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.120966 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:12.121050 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 04:44:12.121142 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":137,"threads_started":1}
I20260809 04:44:12.121569 26236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.114921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:38507 (local address 127.0.0.1:45686)
0809 04:44:12.115334 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:12.115388 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:12.115510 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:12.116795 (+ 1285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:12.116801 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:12.116824 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:12.116972 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:12.116980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.118473 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.118476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:12.119778 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:12.119789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.120791 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.120795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:12.120919 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.120943 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:44:12.121022 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 04:44:12.121098 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":39,"thread_start_us":104,"threads_started":1}
I20260809 04:44:12.121614 26239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.115786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.5.222.193:38135)
0809 04:44:12.116582 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:12.116593 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:12.116617 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:44:12.116697 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:12.116704 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:12.116871 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:12.117055 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:12.117295 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.118024 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.119640 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.120382 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.120513 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.120709 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:12.121360 (+ 651us) server_negotiation.cc:299] Negotiation successful
0809 04:44:12.121449 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":158,"threads_started":1}
I20260809 04:44:12.121690 26241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.115959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.5.222.194:37079)
0809 04:44:12.116658 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:12.116666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:12.116684 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:44:12.116783 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:12.116788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:12.116966 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:12.117393 (+ 427us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:12.117517 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.118582 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.119961 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.120503 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.120907 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.121070 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:12.121119 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 04:44:12.121255 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":131,"threads_started":1}
I20260809 04:44:12.121935 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.115473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:38135 (local address 127.0.0.1:35674)
0809 04:44:12.116386 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:12.116397 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:12.116470 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:12.117033 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:12.117036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:12.117055 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:12.117188 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:12.117199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.118151 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.118157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:12.119514 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:12.119524 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.120529 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.120536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:12.120654 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.120700 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 04:44:12.120806 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 04:44:12.121816 (+ 1010us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"mutex_wait_us":92,"thread_start_us":103,"threads_started":1}
I20260809 04:44:12.124487 25873 tablet_service.cc:1467] Processing CreateTablet for tablet ff7917fabbe845088f229378f4fa452f (DEFAULT_TABLE table=test-table [id=c5ef8fbcd5634270a411c8b67fda2660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:12.124576 26153 tablet_service.cc:1467] Processing CreateTablet for tablet ff7917fabbe845088f229378f4fa452f (DEFAULT_TABLE table=test-table [id=c5ef8fbcd5634270a411c8b67fda2660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:12.124576 26014 tablet_service.cc:1467] Processing CreateTablet for tablet ff7917fabbe845088f229378f4fa452f (DEFAULT_TABLE table=test-table [id=c5ef8fbcd5634270a411c8b67fda2660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:12.125564 25873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff7917fabbe845088f229378f4fa452f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:12.125597 26014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff7917fabbe845088f229378f4fa452f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:12.125597 26153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff7917fabbe845088f229378f4fa452f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:12.132586 26243 tablet_bootstrap.cc:492] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3: Bootstrap starting.
I20260809 04:44:12.132803 26242 tablet_bootstrap.cc:492] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162: Bootstrap starting.
I20260809 04:44:12.133083 26244 tablet_bootstrap.cc:492] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa: Bootstrap starting.
I20260809 04:44:12.134917 26243 tablet_bootstrap.cc:654] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:12.135105 26244 tablet_bootstrap.cc:654] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:12.135106 26242 tablet_bootstrap.cc:654] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:12.135706 26243 log.cc:824] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:12.135922 26244 log.cc:824] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:12.135922 26242 log.cc:824] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:12.137902 26243 tablet_bootstrap.cc:492] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3: No bootstrap required, opened a new log
I20260809 04:44:12.137986 26242 tablet_bootstrap.cc:492] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162: No bootstrap required, opened a new log
I20260809 04:44:12.138105 26244 tablet_bootstrap.cc:492] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa: No bootstrap required, opened a new log
I20260809 04:44:12.138119 26243 ts_tablet_manager.cc:1397] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260809 04:44:12.138175 26242 ts_tablet_manager.cc:1397] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:44:12.138255 26244 ts_tablet_manager.cc:1397] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:44:12.141423 26243 raft_consensus.cc:348] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f43ec14442f4e8cadafba82dc7feca3" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38507 } } peers { permanent_uuid: "dd76778620cd4599adfb01859440c3aa" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 37079 } } peers { permanent_uuid: "3912dec245944d4cb2156381bde24162" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38135 } }
I20260809 04:44:12.141654 26243 raft_consensus.cc:374] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:12.141707 26243 raft_consensus.cc:729] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f43ec14442f4e8cadafba82dc7feca3, State: Initialized, Role: FOLLOWER
I20260809 04:44:12.142127 26243 consensus_queue.cc:260] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f43ec14442f4e8cadafba82dc7feca3" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38507 } } peers { permanent_uuid: "dd76778620cd4599adfb01859440c3aa" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 37079 } } peers { permanent_uuid: "3912dec245944d4cb2156381bde24162" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38135 } }
I20260809 04:44:12.142236 26242 raft_consensus.cc:348] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f43ec14442f4e8cadafba82dc7feca3" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38507 } } peers { permanent_uuid: "dd76778620cd4599adfb01859440c3aa" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 37079 } } peers { permanent_uuid: "3912dec245944d4cb2156381bde24162" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38135 } }
I20260809 04:44:12.142236 26244 raft_consensus.cc:348] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f43ec14442f4e8cadafba82dc7feca3" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38507 } } peers { permanent_uuid: "dd76778620cd4599adfb01859440c3aa" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 37079 } } peers { permanent_uuid: "3912dec245944d4cb2156381bde24162" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38135 } }
I20260809 04:44:12.142488 26242 raft_consensus.cc:374] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:12.142488 26244 raft_consensus.cc:374] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:12.142549 26244 raft_consensus.cc:729] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd76778620cd4599adfb01859440c3aa, State: Initialized, Role: FOLLOWER
I20260809 04:44:12.142549 26242 raft_consensus.cc:729] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3912dec245944d4cb2156381bde24162, State: Initialized, Role: FOLLOWER
I20260809 04:44:12.142935 26224 heartbeater.cc:503] Master 127.5.222.254:46869 was elected leader, sending a full tablet report...
I20260809 04:44:12.143013 26242 consensus_queue.cc:260] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f43ec14442f4e8cadafba82dc7feca3" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38507 } } peers { permanent_uuid: "dd76778620cd4599adfb01859440c3aa" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 37079 } } peers { permanent_uuid: "3912dec245944d4cb2156381bde24162" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38135 } }
I20260809 04:44:12.143148 26243 ts_tablet_manager.cc:1428] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260809 04:44:12.143671 25944 heartbeater.cc:503] Master 127.5.222.254:46869 was elected leader, sending a full tablet report...
I20260809 04:44:12.143920 26242 ts_tablet_manager.cc:1428] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:44:12.144380 26244 consensus_queue.cc:260] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f43ec14442f4e8cadafba82dc7feca3" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38507 } } peers { permanent_uuid: "dd76778620cd4599adfb01859440c3aa" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 37079 } } peers { permanent_uuid: "3912dec245944d4cb2156381bde24162" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38135 } }
I20260809 04:44:12.145399 26244 ts_tablet_manager.cc:1428] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 04:44:12.145748 26083 heartbeater.cc:503] Master 127.5.222.254:46869 was elected leader, sending a full tablet report...
W20260809 04:44:12.169596 25945 tablet.cc:2369] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:12.270556 26249 raft_consensus.cc:482] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:12.270859 26249 raft_consensus.cc:504] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f43ec14442f4e8cadafba82dc7feca3" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38507 } } peers { permanent_uuid: "dd76778620cd4599adfb01859440c3aa" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 37079 } } peers { permanent_uuid: "3912dec245944d4cb2156381bde24162" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38135 } }
I20260809 04:44:12.272063 26249 leader_election.cc:288] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f43ec14442f4e8cadafba82dc7feca3 (127.5.222.195:38507), dd76778620cd4599adfb01859440c3aa (127.5.222.194:37079)
I20260809 04:44:12.276104 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.272415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:35603 (local address 127.5.222.195:38507)
0809 04:44:12.272609 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:12.272615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:12.272727 (+ 112us) server_negotiation.cc:407] Connection header received
0809 04:44:12.272917 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:12.272926 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:12.272978 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:12.273040 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:12.273460 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.273940 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.275080 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.275490 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.275619 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.275678 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:12.275866 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 04:44:12.275949 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 04:44:12.276129 26241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.272449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:49403 (local address 127.5.222.194:37079)
0809 04:44:12.272667 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:12.272673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:12.272786 (+ 113us) server_negotiation.cc:407] Connection header received
0809 04:44:12.272943 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:12.272945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:12.272988 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:12.273082 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:12.273426 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.273930 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.275121 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.275507 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.275640 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.275690 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:12.275918 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 04:44:12.276022 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260809 04:44:12.276153 26252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.272301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:38507 (local address 127.5.222.193:35603)
0809 04:44:12.272610 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:12.272626 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:12.272785 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:12.273084 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:12.273087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:12.273103 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:12.273308 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:12.273314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.274082 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.274086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:12.274950 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:12.274959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.275619 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.275623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:12.275725 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.275748 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:44:12.275817 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 04:44:12.275905 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":20,"spinlock_wait_cycles":21248,"thread_start_us":96,"threads_started":1}
I20260809 04:44:12.276155 26251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.272321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:37079 (local address 127.5.222.193:49403)
0809 04:44:12.272652 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:12.272673 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:12.272804 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:12.273117 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:12.273119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:12.273172 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:12.273304 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:12.273309 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.274052 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.274058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:12.274961 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:12.274970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.275665 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.275669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:12.275766 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.275783 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 04:44:12.275883 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 04:44:12.275948 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":82,"thread_start_us":103,"threads_started":1}
I20260809 04:44:12.276844 26173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff7917fabbe845088f229378f4fa452f" candidate_uuid: "3912dec245944d4cb2156381bde24162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f43ec14442f4e8cadafba82dc7feca3" is_pre_election: true
I20260809 04:44:12.276944 26034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff7917fabbe845088f229378f4fa452f" candidate_uuid: "3912dec245944d4cb2156381bde24162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd76778620cd4599adfb01859440c3aa" is_pre_election: true
I20260809 04:44:12.277235 26173 raft_consensus.cc:2454] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3912dec245944d4cb2156381bde24162 in term 0.
I20260809 04:44:12.277315 26034 raft_consensus.cc:2454] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3912dec245944d4cb2156381bde24162 in term 0.
I20260809 04:44:12.277727 25825 leader_election.cc:302] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3912dec245944d4cb2156381bde24162, 5f43ec14442f4e8cadafba82dc7feca3; no voters:
I20260809 04:44:12.277958 26249 raft_consensus.cc:2781] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:12.278065 26249 raft_consensus.cc:482] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:12.278128 26249 raft_consensus.cc:3037] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:12.279175 26249 raft_consensus.cc:504] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f43ec14442f4e8cadafba82dc7feca3" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38507 } } peers { permanent_uuid: "dd76778620cd4599adfb01859440c3aa" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 37079 } } peers { permanent_uuid: "3912dec245944d4cb2156381bde24162" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38135 } }
I20260809 04:44:12.279557 26249 leader_election.cc:288] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [CANDIDATE]: Term 1 election: Requested vote from peers 5f43ec14442f4e8cadafba82dc7feca3 (127.5.222.195:38507), dd76778620cd4599adfb01859440c3aa (127.5.222.194:37079)
I20260809 04:44:12.279888 26173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff7917fabbe845088f229378f4fa452f" candidate_uuid: "3912dec245944d4cb2156381bde24162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f43ec14442f4e8cadafba82dc7feca3"
I20260809 04:44:12.279917 26034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff7917fabbe845088f229378f4fa452f" candidate_uuid: "3912dec245944d4cb2156381bde24162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd76778620cd4599adfb01859440c3aa"
I20260809 04:44:12.280001 26173 raft_consensus.cc:3037] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:12.280068 26034 raft_consensus.cc:3037] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:12.281025 26173 raft_consensus.cc:2454] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3912dec245944d4cb2156381bde24162 in term 1.
I20260809 04:44:12.281200 26034 raft_consensus.cc:2454] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3912dec245944d4cb2156381bde24162 in term 1.
I20260809 04:44:12.281343 25825 leader_election.cc:302] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3912dec245944d4cb2156381bde24162, 5f43ec14442f4e8cadafba82dc7feca3; no voters:
I20260809 04:44:12.281543 26249 raft_consensus.cc:2781] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:12.281792 26249 raft_consensus.cc:686] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [term 1 LEADER]: Becoming Leader. State: Replica: 3912dec245944d4cb2156381bde24162, State: Running, Role: LEADER
I20260809 04:44:12.282109 26249 consensus_queue.cc:237] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f43ec14442f4e8cadafba82dc7feca3" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38507 } } peers { permanent_uuid: "dd76778620cd4599adfb01859440c3aa" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 37079 } } peers { permanent_uuid: "3912dec245944d4cb2156381bde24162" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38135 } }
I20260809 04:44:12.285471 25743 catalog_manager.cc:5495] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3912dec245944d4cb2156381bde24162 (127.5.222.193). New cstate: current_term: 1 leader_uuid: "3912dec245944d4cb2156381bde24162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f43ec14442f4e8cadafba82dc7feca3" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd76778620cd4599adfb01859440c3aa" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 37079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3912dec245944d4cb2156381bde24162" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 38135 } health_report { overall_health: HEALTHY } } }
I20260809 04:44:12.311522 26239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.308032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35690 (local address 127.5.222.193:38135)
0809 04:44:12.308184 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:12.308188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:12.308217 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:44:12.308297 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:12.308300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:12.308362 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:12.308463 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:12.308720 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.309340 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.310155 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:12.310383 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:12.310466 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:12.310752 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:12.311161 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:12.311271 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:12.311298 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 04:44:12.311343 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:44:12.317828 26034 raft_consensus.cc:1261] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa [term 1 FOLLOWER]: Refusing update from remote peer 3912dec245944d4cb2156381bde24162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:12.317833 26173 raft_consensus.cc:1261] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3 [term 1 FOLLOWER]: Refusing update from remote peer 3912dec245944d4cb2156381bde24162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:12.318589 26253 consensus_queue.cc:1035] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd76778620cd4599adfb01859440c3aa" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 37079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:12.318801 26249 consensus_queue.cc:1035] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f43ec14442f4e8cadafba82dc7feca3" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:12.336414 25743 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:12.336602 25743 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:12.339104 25743 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fe6d761665645ee9168cc8ebec59a26: Sending DeleteTablet for 3 replicas of tablet ff7917fabbe845088f229378f4fa452f
I20260809 04:44:12.339956 26153 tablet_service.cc:1514] Processing DeleteTablet for tablet ff7917fabbe845088f229378f4fa452f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:12 UTC) from {username='slave'} at 127.0.0.1:45686
I20260809 04:44:12.339951 25873 tablet_service.cc:1514] Processing DeleteTablet for tablet ff7917fabbe845088f229378f4fa452f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:12 UTC) from {username='slave'} at 127.0.0.1:35674
I20260809 04:44:12.340075 26014 tablet_service.cc:1514] Processing DeleteTablet for tablet ff7917fabbe845088f229378f4fa452f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:12 UTC) from {username='slave'} at 127.0.0.1:50086
I20260809 04:44:12.340463 26268 tablet_replica.cc:330] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162: stopping tablet replica
I20260809 04:44:12.340493 26269 tablet_replica.cc:330] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa: stopping tablet replica
I20260809 04:44:12.340682 26267 tablet_replica.cc:330] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3: stopping tablet replica
I20260809 04:44:12.340785 26268 raft_consensus.cc:2229] T ff7917fabbe845088f229378f4fa452f P 3912dec245944d4cb2156381bde24162 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:12.340876 26269 raft_consensus.cc:2229] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:12.341013 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 25810
I20260809 04:44:12.341289 26267 raft_consensus.cc:2229] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:12.341629 26267 raft_consensus.cc:2258] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:12.344035 26269 raft_consensus.cc:2258] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:12.345188 26267 ts_tablet_manager.cc:1905] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:12.345585 26269 ts_tablet_manager.cc:1905] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:12.348075 26267 ts_tablet_manager.cc:1918] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:12.348184 26267 log.cc:1196] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/ff7917fabbe845088f229378f4fa452f
I20260809 04:44:12.348491 26269 ts_tablet_manager.cc:1918] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:12.348537 26267 ts_tablet_manager.cc:1939] T ff7917fabbe845088f229378f4fa452f P 5f43ec14442f4e8cadafba82dc7feca3: Deleting consensus metadata
I20260809 04:44:12.348591 26269 log.cc:1196] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/ff7917fabbe845088f229378f4fa452f
I20260809 04:44:12.348910 26269 ts_tablet_manager.cc:1939] T ff7917fabbe845088f229378f4fa452f P dd76778620cd4599adfb01859440c3aa: Deleting consensus metadata
W20260809 04:44:12.349745 25729 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:38135 (error 108)
I20260809 04:44:12.349841 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 25947
I20260809 04:44:12.351842 25730 catalog_manager.cc:4841] TS dd76778620cd4599adfb01859440c3aa (127.5.222.194:37079): tablet ff7917fabbe845088f229378f4fa452f (table test-table [id=c5ef8fbcd5634270a411c8b67fda2660]) successfully deleted
I20260809 04:44:12.352044 25728 catalog_manager.cc:4841] TS 5f43ec14442f4e8cadafba82dc7feca3 (127.5.222.195:38507): tablet ff7917fabbe845088f229378f4fa452f (table test-table [id=c5ef8fbcd5634270a411c8b67fda2660]) successfully deleted
I20260809 04:44:12.353276 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:12.352879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:38135 (local address 127.0.0.1:35704)
0809 04:44:12.353057 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:12.353141 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:38135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 04:44:12.353535 25729 catalog_manager.cc:4584] TS 3912dec245944d4cb2156381bde24162 (127.5.222.193:38135): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff7917fabbe845088f229378f4fa452f: Network error: Client connection negotiation failed: client connection to 127.5.222.193:38135: connect: Connection refused (error 111)
I20260809 04:44:12.358342 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 26086
I20260809 04:44:12.366421 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 25710
2026-08-09T04:44:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1221 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:12 dist-test-slave-vxj2 krb5kdc[26274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:12 dist-test-slave-vxj2 krb5kdc[26274](info): set up 2 sockets
Aug 09 04:44:12 dist-test-slave-vxj2 krb5kdc[26274](info): commencing operation
krb5kdc: starting...
W20260809 04:44:13.462317 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:13 dist-test-slave-vxj2 krb5kdc[26274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:44:13.592521 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:43493
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:43493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:13.708940 26288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:13.709306 26288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:13.709401 26288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:13.713474 26288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:13.713590 26288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:13.713635 26288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:13.713703 26288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:13.713749 26288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:13.719166 26288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:43493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:43493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:13.720320 26288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:13.721431 26288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:13.730180 26297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:13.730172 26295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:13.730172 26293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:13.730437 26288 server_base.cc:1034] running on GCE node
I20260809 04:44:13.731096 26288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:13.732208 26288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:13.733392 26288 hybrid_clock.cc:648] HybridClock initialized: now 1786250653733365 us; error 39 us; skew 500 ppm
Aug 09 04:44:13 dist-test-slave-vxj2 krb5kdc[26274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250653, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:13.736172 26288 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:44:13.737010 26288 webserver.cc:460] Webserver started at http://127.5.222.254:39599/ using document root <none> and password file <none>
I20260809 04:44:13.737584 26288 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:13.737646 26288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:13.737818 26288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:13.739925 26288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "fc5f1c33c10444a6b21c67f8c1ae8060"
format_stamp: "Formatted at 2026-08-09 04:44:13 on dist-test-slave-vxj2"
server_key: "a2730f9cc803c8b2ccd7e1903e54aade"
server_key_iv: "31e0ce3308ea28f50bb242aa240916a3"
server_key_version: "encryptionkey@0"
I20260809 04:44:13.740480 26288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "fc5f1c33c10444a6b21c67f8c1ae8060"
format_stamp: "Formatted at 2026-08-09 04:44:13 on dist-test-slave-vxj2"
server_key: "a2730f9cc803c8b2ccd7e1903e54aade"
server_key_iv: "31e0ce3308ea28f50bb242aa240916a3"
server_key_version: "encryptionkey@0"
I20260809 04:44:13.744455 26288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:44:13.747044 26305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:13.748260 26288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:44:13.748360 26288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "fc5f1c33c10444a6b21c67f8c1ae8060"
format_stamp: "Formatted at 2026-08-09 04:44:13 on dist-test-slave-vxj2"
server_key: "a2730f9cc803c8b2ccd7e1903e54aade"
server_key_iv: "31e0ce3308ea28f50bb242aa240916a3"
server_key_version: "encryptionkey@0"
I20260809 04:44:13.748447 26288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:13.769611 26288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:13.772918 26288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:13.773087 26288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:13.780361 26288 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:43493
I20260809 04:44:13.780455 26361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:43493 every 8 connection(s)
I20260809 04:44:13.781401 26288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:13.784163 26362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:13.789696 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 26288
I20260809 04:44:13.789702 26362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060: Bootstrap starting.
I20260809 04:44:13.789805 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:13.790144 6011 external_mini_cluster.cc:1428] Setting key 885925b6e229e298e6fdcbba147e80f4
I20260809 04:44:13.792284 26362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:13.793067 26362 log.cc:824] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:13.795202 26362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060: No bootstrap required, opened a new log
Aug 09 04:44:13 dist-test-slave-vxj2 krb5kdc[26274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:13.800092 26362 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5f1c33c10444a6b21c67f8c1ae8060" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43493 } }
I20260809 04:44:13.800357 26362 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:13.800460 26362 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc5f1c33c10444a6b21c67f8c1ae8060, State: Initialized, Role: FOLLOWER
I20260809 04:44:13.801040 26362 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5f1c33c10444a6b21c67f8c1ae8060" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43493 } }
I20260809 04:44:13.801260 26362 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:13.801352 26362 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:13.801473 26362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:13.802464 26362 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5f1c33c10444a6b21c67f8c1ae8060" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43493 } }
I20260809 04:44:13.802861 26362 leader_election.cc:302] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc5f1c33c10444a6b21c67f8c1ae8060; no voters:
I20260809 04:44:13.803289 26362 leader_election.cc:288] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:13.803398 26368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:13.803673 26368 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [term 1 LEADER]: Becoming Leader. State: Replica: fc5f1c33c10444a6b21c67f8c1ae8060, State: Running, Role: LEADER
I20260809 04:44:13.803985 26368 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5f1c33c10444a6b21c67f8c1ae8060" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43493 } }
I20260809 04:44:13.804733 26362 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:13.805224 26368 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc5f1c33c10444a6b21c67f8c1ae8060. Latest consensus state: current_term: 1 leader_uuid: "fc5f1c33c10444a6b21c67f8c1ae8060" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5f1c33c10444a6b21c67f8c1ae8060" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43493 } } }
I20260809 04:44:13.805347 26368 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:13.805670 26370 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc5f1c33c10444a6b21c67f8c1ae8060" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5f1c33c10444a6b21c67f8c1ae8060" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43493 } } }
I20260809 04:44:13.805754 26370 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:13.805841 26372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:13.806254 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:13.791533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33972 (local address 127.5.222.254:43493)
0809 04:44:13.792003 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:13.792018 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:13.792053 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:44:13.792730 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:13.792748 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:13.793075 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:13.793486 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:13.794334 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:13.795341 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:13.796577 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:13.797057 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:13.797298 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:13.800199 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:13.800204 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:13.800217 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:13.800245 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:13.802174 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:13.802662 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:13.802668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:13.802674 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:13.802828 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:13.803205 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:13.803211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:13.803216 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:13.803771 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:13.804648 (+ 877us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:13.804820 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 04:44:13.805138 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":151,"threads_started":1}
I20260809 04:44:13.808990 26372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:13.814268 26372 catalog_manager.cc:1347] Generated new cluster ID: 1d9544e793d34a5f815264f38b35380a
I20260809 04:44:13.814342 26372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:13.827162 26372 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:13.828397 26372 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:13.832854 26372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060: Generated new TSK 0
I20260809 04:44:13.833586 26372 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:44:13.863732 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43493
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:13.975569 26392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:13.975824 26392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:13.975867 26392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:13.979948 26392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:13.980005 26392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:13.980095 26392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:44:13.985301 26392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:13.986367 26392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:13.987416 26392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:13.996897 26400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:13.996927 26398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:13.996892 26397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:13.997478 26392 server_base.cc:1034] running on GCE node
I20260809 04:44:13.997835 26392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:13.998392 26392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:13.999557 26392 hybrid_clock.cc:648] HybridClock initialized: now 1786250653999522 us; error 45 us; skew 500 ppm
Aug 09 04:44:14 dist-test-slave-vxj2 krb5kdc[26274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250654, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:14.008980 26392 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:44:14.010064 26392 webserver.cc:460] Webserver started at http://127.5.222.193:36041/ using document root <none> and password file <none>
I20260809 04:44:14.010684 26392 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:14.010779 26392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:14.011026 26392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:14.013310 26392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "9731309bbf1a4886898350731686e934"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-vxj2"
server_key: "c924066cc016a648c8cbec07c7db24aa"
server_key_iv: "6cba01371ea02aa21d9ea1f093bc1885"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.013993 26392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "9731309bbf1a4886898350731686e934"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-vxj2"
server_key: "c924066cc016a648c8cbec07c7db24aa"
server_key_iv: "6cba01371ea02aa21d9ea1f093bc1885"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.018659 26392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:44:14.021297 26408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.022547 26392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:14.022691 26392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "9731309bbf1a4886898350731686e934"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-vxj2"
server_key: "c924066cc016a648c8cbec07c7db24aa"
server_key_iv: "6cba01371ea02aa21d9ea1f093bc1885"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.022806 26392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:14.034210 26392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:14.037125 26392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:14.037384 26392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:14.037988 26392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:14.038841 26392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:14.038913 26392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.038977 26392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:14.039019 26392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.048658 26392 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:42539
I20260809 04:44:14.048713 26527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:42539 every 8 connection(s)
I20260809 04:44:14.049651 26392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:14.050719 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 26392
I20260809 04:44:14.050879 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:14.051301 6011 external_mini_cluster.cc:1428] Setting key e30e2c46ea3c8c62e2e1c62dedf10e80
Aug 09 04:44:14 dist-test-slave-vxj2 krb5kdc[26274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250654, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
I20260809 04:44:14.063946 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.051764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:57507 (local address 127.5.222.254:43493)
0809 04:44:14.051937 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:14.051942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:14.052863 (+ 921us) server_negotiation.cc:407] Connection header received
0809 04:44:14.054237 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:14.054241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:14.054295 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:14.054375 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:14.055510 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.056061 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.057261 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.057518 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.057633 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.060081 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:14.060084 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:14.060086 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:14.060110 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:14.061693 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:14.062191 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:14.062195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:14.062197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:14.062277 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:14.062747 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:14.062752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:14.062754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:14.062939 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:14.063046 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:14.063615 (+ 569us) server_negotiation.cc:299] Negotiation successful
0809 04:44:14.063755 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
I20260809 04:44:14.064852 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.052209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43493 (local address 127.5.222.193:57507)
0809 04:44:14.052715 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:14.052768 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:14.054033 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:14.054547 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:14.054554 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:14.054901 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:14.055352 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.055365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.056193 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.056196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.057097 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.057106 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.057684 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.057688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.057853 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.058455 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:14.058470 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:14.059913 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:14.061804 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:14.061810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:14.061823 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:14.062084 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:14.062370 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:14.062373 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:14.062375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:14.062635 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:14.063046 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:14.063054 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:14.063291 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 04:44:14.063570 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 04:44:14.063911 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":143,"threads_started":1}
I20260809 04:44:14.066152 26528 heartbeater.cc:347] Connected to a master server at 127.5.222.254:43493
I20260809 04:44:14.066425 26528 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:14.066956 26528 heartbeater.cc:511] Master 127.5.222.254:43493 requested a full tablet report, sending...
I20260809 04:44:14.068696 26322 ts_manager.cc:194] Registered new tserver with Master: 9731309bbf1a4886898350731686e934 (127.5.222.193:42539)
I20260809 04:44:14.070672 26322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:57507
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:44:14.081782 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43493
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:14.191941 26534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:14.192183 26534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:14.192262 26534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:14.197085 26534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:14.197182 26534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:14.197297 26534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:44:14.202479 26534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:14.203464 26534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:14.204437 26534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:14.214082 26540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:14.214083 26539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:14.214087 26542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:14.214504 26534 server_base.cc:1034] running on GCE node
I20260809 04:44:14.214898 26534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:14.215591 26534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:14.216775 26534 hybrid_clock.cc:648] HybridClock initialized: now 1786250654216745 us; error 50 us; skew 500 ppm
Aug 09 04:44:14 dist-test-slave-vxj2 krb5kdc[26274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250654, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:14.219611 26534 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:44:14.220417 26534 webserver.cc:460] Webserver started at http://127.5.222.194:38235/ using document root <none> and password file <none>
I20260809 04:44:14.220958 26534 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:14.221019 26534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:14.221268 26534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:14.223266 26534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "7d25059fb52d41849dd8cacb84c88974"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-vxj2"
server_key: "6fee36e4b6e6f60fe8c6910fdcd49c38"
server_key_iv: "603864ff1ae4330bdc602433ce790a9a"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.223780 26534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "7d25059fb52d41849dd8cacb84c88974"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-vxj2"
server_key: "6fee36e4b6e6f60fe8c6910fdcd49c38"
server_key_iv: "603864ff1ae4330bdc602433ce790a9a"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.227713 26534 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 04:44:14.230106 26550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.231391 26534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:14.231531 26534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "7d25059fb52d41849dd8cacb84c88974"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-vxj2"
server_key: "6fee36e4b6e6f60fe8c6910fdcd49c38"
server_key_iv: "603864ff1ae4330bdc602433ce790a9a"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.231654 26534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:14.260875 26534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:14.264345 26534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:14.264577 26534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:14.265228 26534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:14.266090 26534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:14.266163 26534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.266242 26534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:14.266285 26534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.275411 26534 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:43987
I20260809 04:44:14.275440 26668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:43987 every 8 connection(s)
I20260809 04:44:14.276304 26534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:14.278677 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 26534
I20260809 04:44:14.278867 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:14.279242 6011 external_mini_cluster.cc:1428] Setting key 45c41cce9cccdc25c2ecbb25f6feb612
Aug 09 04:44:14 dist-test-slave-vxj2 krb5kdc[26274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250654, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
I20260809 04:44:14.290892 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.278127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:58503 (local address 127.5.222.254:43493)
0809 04:44:14.278267 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:14.278289 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:14.279161 (+ 872us) server_negotiation.cc:407] Connection header received
0809 04:44:14.280269 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:14.280273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:14.280319 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:14.280393 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:14.281670 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.282520 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.284287 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.284529 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.284645 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.287162 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:14.287166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:14.287169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:14.287203 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:14.288644 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:14.289259 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:14.289262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:14.289264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:14.289352 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:14.289713 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:14.289716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:14.289718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:14.289905 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:14.290026 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:14.290587 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 04:44:14.290711 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":52}
I20260809 04:44:14.291790 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.278444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43493 (local address 127.5.222.194:58503)
0809 04:44:14.278962 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:14.279030 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:14.280040 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:14.280604 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:14.280613 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:14.280971 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:14.281519 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.281533 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.282675 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.282680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.284153 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.284164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.284686 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.284690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.284866 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.285529 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:14.285545 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:14.286995 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:14.288775 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:14.288781 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:14.288796 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:14.289132 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:14.289464 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:14.289467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:14.289469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:14.289606 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:14.290023 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:14.290029 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:14.290299 (+ 270us) client_negotiation.cc:769] Sending connection context
0809 04:44:14.290498 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 04:44:14.290822 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":149,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
I20260809 04:44:14.293008 26669 heartbeater.cc:347] Connected to a master server at 127.5.222.254:43493
I20260809 04:44:14.293313 26669 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:14.293907 26669 heartbeater.cc:511] Master 127.5.222.254:43493 requested a full tablet report, sending...
I20260809 04:44:14.295128 26322 ts_manager.cc:194] Registered new tserver with Master: 7d25059fb52d41849dd8cacb84c88974 (127.5.222.194:43987)
I20260809 04:44:14.295802 26322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:58503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:44:14.308300 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43493
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:14.427500 26675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:14.427773 26675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:14.427846 26675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:14.431881 26675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:14.431975 26675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:14.432065 26675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:44:14.437539 26675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:43493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:14.438603 26675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:14.439613 26675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:14.449218 26682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:14.449213 26684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:14.449359 26675 server_base.cc:1034] running on GCE node
W20260809 04:44:14.449292 26681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:14.449965 26675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:14.450567 26675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:14.451743 26675 hybrid_clock.cc:648] HybridClock initialized: now 1786250654451730 us; error 25 us; skew 500 ppm
Aug 09 04:44:14 dist-test-slave-vxj2 krb5kdc[26274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250654, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:14.454563 26675 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:44:14.455542 26675 webserver.cc:460] Webserver started at http://127.5.222.195:33233/ using document root <none> and password file <none>
I20260809 04:44:14.456097 26675 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:14.456185 26675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:14.456440 26675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:14.458482 26675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "05c90d307b174f4381f2756f544d8072"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-vxj2"
server_key: "436e3672ef66161e130eb9751c7886f1"
server_key_iv: "c38778b31f1c5013697a1eb22b1f96d1"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.459038 26675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "05c90d307b174f4381f2756f544d8072"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-vxj2"
server_key: "436e3672ef66161e130eb9751c7886f1"
server_key_iv: "c38778b31f1c5013697a1eb22b1f96d1"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.462580 26675 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:44:14.464781 26691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.466007 26675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:44:14.466149 26675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "05c90d307b174f4381f2756f544d8072"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-vxj2"
server_key: "436e3672ef66161e130eb9751c7886f1"
server_key_iv: "c38778b31f1c5013697a1eb22b1f96d1"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.466254 26675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:14.476219 26675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:14.479583 26675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:14.479796 26675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:14.480383 26675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:14.481271 26675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:14.481338 26675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.481407 26675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:14.481462 26675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.490124 26675 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:36803
I20260809 04:44:14.490151 26813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:36803 every 8 connection(s)
I20260809 04:44:14.490974 26675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:14.495146 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 26675
I20260809 04:44:14.495301 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:14.495579 6011 external_mini_cluster.cc:1428] Setting key 69441c58c54c3c343924935f3652acdb
Aug 09 04:44:14 dist-test-slave-vxj2 krb5kdc[26274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250654, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:14.504722 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.492737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:33373 (local address 127.5.222.254:43493)
0809 04:44:14.492876 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:14.492879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:14.493724 (+ 845us) server_negotiation.cc:407] Connection header received
0809 04:44:14.495001 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:14.495004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:14.495063 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:14.495145 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:14.496502 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.497309 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.498602 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.498825 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.498908 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.501380 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:14.501389 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:14.501391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:14.501415 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:14.502620 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:14.503148 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:14.503153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:14.503155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:14.503240 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:14.503596 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:14.503600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:14.503601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:14.503783 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:14.503894 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:14.504427 (+ 533us) server_negotiation.cc:299] Negotiation successful
0809 04:44:14.504552 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:44:14.505645 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.493103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:43493 (local address 127.5.222.195:33373)
0809 04:44:14.493584 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:14.493624 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:14.494789 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:14.495334 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:14.495341 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:14.495747 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:14.496298 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.496314 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.497444 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.497447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.498490 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.498501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.498956 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.498958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.499090 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.499749 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:14.499766 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:14.501222 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:14.502764 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:14.502770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:14.502783 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:14.503040 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:14.503344 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:14.503347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:14.503349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:14.503494 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:14.503893 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:14.503899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:14.504129 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 04:44:14.504369 (+ 240us) client_negotiation.cc:240] Negotiation successful
0809 04:44:14.504685 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":119,"threads_started":1}
I20260809 04:44:14.506738 26814 heartbeater.cc:347] Connected to a master server at 127.5.222.254:43493
I20260809 04:44:14.506946 26814 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:14.507366 26814 heartbeater.cc:511] Master 127.5.222.254:43493 requested a full tablet report, sending...
I20260809 04:44:14.508309 26322 ts_manager.cc:194] Registered new tserver with Master: 05c90d307b174f4381f2756f544d8072 (127.5.222.195:36803)
I20260809 04:44:14.508915 26322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:33373
I20260809 04:44:14.509624 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:44:14.518971 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.511659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38548 (local address 127.5.222.254:43493)
0809 04:44:14.511775 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:14.511779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:14.511952 (+ 173us) server_negotiation.cc:407] Connection header received
0809 04:44:14.512057 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:14.512059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:14.512094 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:14.512153 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:14.512785 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.513323 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.514122 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.514362 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.514453 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.515888 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:14.515891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:14.515893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:14.515914 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:14.517415 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:14.517864 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:14.517867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:14.517868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:14.517943 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:14.518210 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:14.518212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:14.518213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:14.518431 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:14.518552 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:14.518775 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 04:44:14.518852 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:44:14.522181 26320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:14.525132 26320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:14.540196 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.533859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:36803 (local address 127.0.0.1:40092)
0809 04:44:14.534880 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:14.534923 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:14.535047 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:14.536230 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:14.536236 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:14.536260 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:14.536339 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.536345 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.537725 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.537729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.538833 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.538844 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.539661 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.539664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.539777 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.539812 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 04:44:14.539920 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 04:44:14.540037 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"thread_start_us":107,"threads_started":1}
I20260809 04:44:14.540195 26830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.534665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40092 (local address 127.5.222.195:36803)
0809 04:44:14.535650 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:14.535663 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:14.535702 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:44:14.535826 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:14.535834 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:14.536013 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:14.536245 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:14.536448 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.537599 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.538976 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.539536 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.539660 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.539813 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:14.539954 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 04:44:14.540046 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":100,"threads_started":1}
I20260809 04:44:14.540658 26826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.533987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:42539 (local address 127.0.0.1:60084)
0809 04:44:14.534868 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:14.534920 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:14.535105 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:14.535910 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:14.535916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:14.535965 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:14.536168 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.536182 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.537683 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.537686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.538883 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.538890 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.540040 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.540043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.540273 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.540349 (+ 76us) client_negotiation.cc:769] Sending connection context
0809 04:44:14.540423 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 04:44:14.540491 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":91,"spinlock_wait_cycles":6272,"thread_start_us":107,"threads_started":1}
I20260809 04:44:14.540969 26831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.534689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53926 (local address 127.5.222.194:43987)
0809 04:44:14.535207 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:14.535220 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:14.535360 (+ 140us) server_negotiation.cc:407] Connection header received
0809 04:44:14.535541 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:14.535547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:14.535735 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:14.535911 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:14.536320 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.537226 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.538829 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.539698 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.539824 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.539969 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:14.540728 (+ 759us) server_negotiation.cc:299] Negotiation successful
0809 04:44:14.540831 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":151,"threads_started":1}
I20260809 04:44:14.541102 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.534185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.5.222.193:42539)
0809 04:44:14.534657 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:14.534675 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:14.535020 (+ 345us) server_negotiation.cc:407] Connection header received
0809 04:44:14.535266 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:14.535273 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:14.535482 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:14.535623 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:14.536309 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.537536 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.539012 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.539760 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.539849 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.539974 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:14.540850 (+ 876us) server_negotiation.cc:299] Negotiation successful
0809 04:44:14.540953 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":101,"threads_started":1}
I20260809 04:44:14.540659 26827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.534243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43987 (local address 127.0.0.1:53926)
0809 04:44:14.535268 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:14.535281 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:14.535367 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:14.535912 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:14.535917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:14.535970 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:14.536165 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.536194 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.537362 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.537365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.538697 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.538708 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.540040 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.540043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.540236 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.540353 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 04:44:14.540423 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 04:44:14.540491 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"mutex_wait_us":157,"spinlock_wait_cycles":29824,"thread_start_us":91,"threads_started":1}
I20260809 04:44:14.543653 26459 tablet_service.cc:1467] Processing CreateTablet for tablet deab857fc98d4367a140dbab1285f250 (DEFAULT_TABLE table=test-table [id=a7361856344a419aadf2de9d604c1dac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:14.543776 26745 tablet_service.cc:1467] Processing CreateTablet for tablet deab857fc98d4367a140dbab1285f250 (DEFAULT_TABLE table=test-table [id=a7361856344a419aadf2de9d604c1dac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:14.544255 26602 tablet_service.cc:1467] Processing CreateTablet for tablet deab857fc98d4367a140dbab1285f250 (DEFAULT_TABLE table=test-table [id=a7361856344a419aadf2de9d604c1dac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:14.544677 26459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deab857fc98d4367a140dbab1285f250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:14.544831 26745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deab857fc98d4367a140dbab1285f250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:14.545229 26602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deab857fc98d4367a140dbab1285f250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:14.551120 26832 tablet_bootstrap.cc:492] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934: Bootstrap starting.
I20260809 04:44:14.552693 26835 tablet_bootstrap.cc:492] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974: Bootstrap starting.
I20260809 04:44:14.553475 26834 tablet_bootstrap.cc:492] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072: Bootstrap starting.
I20260809 04:44:14.553869 26832 tablet_bootstrap.cc:654] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:14.555228 26834 tablet_bootstrap.cc:654] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:14.555258 26832 log.cc:824] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:14.555295 26835 tablet_bootstrap.cc:654] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:14.556080 26835 log.cc:824] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:14.556443 26834 log.cc:824] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:14.557543 26832 tablet_bootstrap.cc:492] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934: No bootstrap required, opened a new log
I20260809 04:44:14.557767 26832 ts_tablet_manager.cc:1397] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 04:44:14.558342 26834 tablet_bootstrap.cc:492] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072: No bootstrap required, opened a new log
I20260809 04:44:14.558539 26834 ts_tablet_manager.cc:1397] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:44:14.558802 26835 tablet_bootstrap.cc:492] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974: No bootstrap required, opened a new log
I20260809 04:44:14.558986 26835 ts_tablet_manager.cc:1397] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:44:14.561422 26834 raft_consensus.cc:348] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c90d307b174f4381f2756f544d8072" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 36803 } } peers { permanent_uuid: "9731309bbf1a4886898350731686e934" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42539 } } peers { permanent_uuid: "7d25059fb52d41849dd8cacb84c88974" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43987 } }
I20260809 04:44:14.561684 26834 raft_consensus.cc:374] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:14.561764 26834 raft_consensus.cc:729] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c90d307b174f4381f2756f544d8072, State: Initialized, Role: FOLLOWER
I20260809 04:44:14.561919 26832 raft_consensus.cc:348] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c90d307b174f4381f2756f544d8072" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 36803 } } peers { permanent_uuid: "9731309bbf1a4886898350731686e934" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42539 } } peers { permanent_uuid: "7d25059fb52d41849dd8cacb84c88974" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43987 } }
I20260809 04:44:14.562160 26832 raft_consensus.cc:374] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:14.562261 26832 raft_consensus.cc:729] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9731309bbf1a4886898350731686e934, State: Initialized, Role: FOLLOWER
I20260809 04:44:14.562285 26834 consensus_queue.cc:260] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c90d307b174f4381f2756f544d8072" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 36803 } } peers { permanent_uuid: "9731309bbf1a4886898350731686e934" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42539 } } peers { permanent_uuid: "7d25059fb52d41849dd8cacb84c88974" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43987 } }
I20260809 04:44:14.562729 26832 consensus_queue.cc:260] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c90d307b174f4381f2756f544d8072" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 36803 } } peers { permanent_uuid: "9731309bbf1a4886898350731686e934" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42539 } } peers { permanent_uuid: "7d25059fb52d41849dd8cacb84c88974" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43987 } }
I20260809 04:44:14.563162 26814 heartbeater.cc:503] Master 127.5.222.254:43493 was elected leader, sending a full tablet report...
I20260809 04:44:14.563400 26834 ts_tablet_manager.cc:1428] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 04:44:14.563740 26832 ts_tablet_manager.cc:1428] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260809 04:44:14.564070 26528 heartbeater.cc:503] Master 127.5.222.254:43493 was elected leader, sending a full tablet report...
I20260809 04:44:14.564381 26835 raft_consensus.cc:348] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c90d307b174f4381f2756f544d8072" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 36803 } } peers { permanent_uuid: "9731309bbf1a4886898350731686e934" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42539 } } peers { permanent_uuid: "7d25059fb52d41849dd8cacb84c88974" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43987 } }
I20260809 04:44:14.564628 26835 raft_consensus.cc:374] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:14.564690 26835 raft_consensus.cc:729] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d25059fb52d41849dd8cacb84c88974, State: Initialized, Role: FOLLOWER
I20260809 04:44:14.565109 26835 consensus_queue.cc:260] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c90d307b174f4381f2756f544d8072" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 36803 } } peers { permanent_uuid: "9731309bbf1a4886898350731686e934" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42539 } } peers { permanent_uuid: "7d25059fb52d41849dd8cacb84c88974" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43987 } }
I20260809 04:44:14.566006 26669 heartbeater.cc:503] Master 127.5.222.254:43493 was elected leader, sending a full tablet report...
I20260809 04:44:14.566088 26835 ts_tablet_manager.cc:1428] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 04:44:14.712489 26840 raft_consensus.cc:482] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:14.712706 26840 raft_consensus.cc:504] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c90d307b174f4381f2756f544d8072" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 36803 } } peers { permanent_uuid: "9731309bbf1a4886898350731686e934" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42539 } } peers { permanent_uuid: "7d25059fb52d41849dd8cacb84c88974" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43987 } }
I20260809 04:44:14.713865 26840 leader_election.cc:288] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05c90d307b174f4381f2756f544d8072 (127.5.222.195:36803), 7d25059fb52d41849dd8cacb84c88974 (127.5.222.194:43987)
I20260809 04:44:14.717892 26831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.714309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:49065 (local address 127.5.222.194:43987)
0809 04:44:14.714457 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:14.714461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:14.714661 (+ 200us) server_negotiation.cc:407] Connection header received
0809 04:44:14.714840 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:14.714843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:14.714890 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:14.714988 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:14.715263 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.715769 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.716856 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.717321 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.717464 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.717522 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:14.717694 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 04:44:14.717761 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:44:14.717909 26844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.714185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:43987 (local address 127.5.222.193:49065)
0809 04:44:14.714554 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:14.714567 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:14.714706 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:14.714995 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:14.714998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:14.715014 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:14.715144 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.715150 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.715883 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.715890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.716724 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.716733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.717446 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.717450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.717552 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.717575 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:44:14.717679 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 04:44:14.717749 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260809 04:44:14.718223 26830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.714310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:52705 (local address 127.5.222.195:36803)
0809 04:44:14.714451 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:14.714457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:14.714763 (+ 306us) server_negotiation.cc:407] Connection header received
0809 04:44:14.714960 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:14.714964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:14.715009 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:14.715104 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:14.715417 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.715917 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.717011 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.717491 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.717631 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.717686 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:14.718041 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 04:44:14.718109 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:44:14.718591 26622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deab857fc98d4367a140dbab1285f250" candidate_uuid: "9731309bbf1a4886898350731686e934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d25059fb52d41849dd8cacb84c88974" is_pre_election: true
I20260809 04:44:14.718786 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.714200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:36803 (local address 127.5.222.193:52705)
0809 04:44:14.714634 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:14.714647 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:14.714823 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:14.715113 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:14.715116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:14.715133 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:14.715286 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.715292 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.716053 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.716060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.716869 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.716878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.717622 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.717627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.717795 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.717814 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 04:44:14.718015 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 04:44:14.718094 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":1,"thread_start_us":134,"threads_started":1}
I20260809 04:44:14.718881 26622 raft_consensus.cc:2454] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9731309bbf1a4886898350731686e934 in term 0.
I20260809 04:44:14.719405 26411 leader_election.cc:302] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d25059fb52d41849dd8cacb84c88974, 9731309bbf1a4886898350731686e934; no voters:
I20260809 04:44:14.719408 26765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deab857fc98d4367a140dbab1285f250" candidate_uuid: "9731309bbf1a4886898350731686e934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c90d307b174f4381f2756f544d8072" is_pre_election: true
I20260809 04:44:14.719686 26840 raft_consensus.cc:2781] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:14.719709 26765 raft_consensus.cc:2454] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9731309bbf1a4886898350731686e934 in term 0.
I20260809 04:44:14.719794 26840 raft_consensus.cc:482] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:14.719839 26840 raft_consensus.cc:3037] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:14.721055 26840 raft_consensus.cc:504] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c90d307b174f4381f2756f544d8072" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 36803 } } peers { permanent_uuid: "9731309bbf1a4886898350731686e934" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42539 } } peers { permanent_uuid: "7d25059fb52d41849dd8cacb84c88974" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43987 } }
I20260809 04:44:14.721488 26840 leader_election.cc:288] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [CANDIDATE]: Term 1 election: Requested vote from peers 05c90d307b174f4381f2756f544d8072 (127.5.222.195:36803), 7d25059fb52d41849dd8cacb84c88974 (127.5.222.194:43987)
I20260809 04:44:14.721823 26622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deab857fc98d4367a140dbab1285f250" candidate_uuid: "9731309bbf1a4886898350731686e934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d25059fb52d41849dd8cacb84c88974"
I20260809 04:44:14.721822 26765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deab857fc98d4367a140dbab1285f250" candidate_uuid: "9731309bbf1a4886898350731686e934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c90d307b174f4381f2756f544d8072"
I20260809 04:44:14.721966 26765 raft_consensus.cc:3037] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:14.721966 26622 raft_consensus.cc:3037] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:14.723033 26765 raft_consensus.cc:2454] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9731309bbf1a4886898350731686e934 in term 1.
I20260809 04:44:14.723132 26622 raft_consensus.cc:2454] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9731309bbf1a4886898350731686e934 in term 1.
I20260809 04:44:14.723340 26410 leader_election.cc:302] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05c90d307b174f4381f2756f544d8072, 9731309bbf1a4886898350731686e934; no voters:
I20260809 04:44:14.723511 26840 raft_consensus.cc:2781] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:14.723771 26840 raft_consensus.cc:686] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [term 1 LEADER]: Becoming Leader. State: Replica: 9731309bbf1a4886898350731686e934, State: Running, Role: LEADER
I20260809 04:44:14.724339 26840 consensus_queue.cc:237] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c90d307b174f4381f2756f544d8072" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 36803 } } peers { permanent_uuid: "9731309bbf1a4886898350731686e934" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42539 } } peers { permanent_uuid: "7d25059fb52d41849dd8cacb84c88974" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43987 } }
I20260809 04:44:14.727941 26319 catalog_manager.cc:5495] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9731309bbf1a4886898350731686e934 (127.5.222.193). New cstate: current_term: 1 leader_uuid: "9731309bbf1a4886898350731686e934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05c90d307b174f4381f2756f544d8072" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 36803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9731309bbf1a4886898350731686e934" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d25059fb52d41849dd8cacb84c88974" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43987 } health_report { overall_health: UNKNOWN } } }
W20260809 04:44:14.741887 26815 tablet.cc:2369] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:14.776620 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.772991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60096 (local address 127.5.222.193:42539)
0809 04:44:14.773141 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:14.773146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:14.773184 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:44:14.773297 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:14.773299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:14.773354 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:14.773470 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:14.773723 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.774236 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.775117 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.775337 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.775450 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.775769 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:14.776179 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:14.776343 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:14.776433 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 04:44:14.776476 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 04:44:14.777423 26670 tablet.cc:2369] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:14.782505 26765 raft_consensus.cc:1261] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072 [term 1 FOLLOWER]: Refusing update from remote peer 9731309bbf1a4886898350731686e934: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:14.782559 26622 raft_consensus.cc:1261] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974 [term 1 FOLLOWER]: Refusing update from remote peer 9731309bbf1a4886898350731686e934: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:14.783221 26840 consensus_queue.cc:1035] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05c90d307b174f4381f2756f544d8072" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 36803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:14.783449 26845 consensus_queue.cc:1035] T deab857fc98d4367a140dbab1285f250 P 9731309bbf1a4886898350731686e934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d25059fb52d41849dd8cacb84c88974" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 43987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:14.790061 26849 mvcc.cc:205] Tried to move back new op lower bound from 7316482681981669376 to 7316482681753841664. Current Snapshot: MvccSnapshot[applied={T|T < 7316482681981669376}]
I20260809 04:44:14.794651 26850 mvcc.cc:205] Tried to move back new op lower bound from 7316482681981669376 to 7316482681753841664. Current Snapshot: MvccSnapshot[applied={T|T < 7316482681981669376}]
I20260809 04:44:14.803086 26322 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:14.803239 26322 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:14.805742 26322 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc5f1c33c10444a6b21c67f8c1ae8060: Sending DeleteTablet for 3 replicas of tablet deab857fc98d4367a140dbab1285f250
I20260809 04:44:14.806603 26459 tablet_service.cc:1514] Processing DeleteTablet for tablet deab857fc98d4367a140dbab1285f250 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:14 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:60084
I20260809 04:44:14.806806 26602 tablet_service.cc:1514] Processing DeleteTablet for tablet deab857fc98d4367a140dbab1285f250 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:14 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:53926
I20260809 04:44:14.806836 26745 tablet_service.cc:1514] Processing DeleteTablet for tablet deab857fc98d4367a140dbab1285f250 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:14 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:40092
I20260809 04:44:14.807483 26859 tablet_replica.cc:330] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974: stopping tablet replica
I20260809 04:44:14.807814 26859 raft_consensus.cc:2229] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:14.808005 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 26392
I20260809 04:44:14.810796 26860 tablet_replica.cc:330] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072: stopping tablet replica
I20260809 04:44:14.810949 26859 raft_consensus.cc:2258] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:14.811115 26860 raft_consensus.cc:2229] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:14.811399 26860 raft_consensus.cc:2258] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:14.812703 26860 ts_tablet_manager.cc:1905] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:14.812731 26859 ts_tablet_manager.cc:1905] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:14.814951 26860 ts_tablet_manager.cc:1918] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:14.815073 26860 log.cc:1196] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/deab857fc98d4367a140dbab1285f250
I20260809 04:44:14.815411 26860 ts_tablet_manager.cc:1939] T deab857fc98d4367a140dbab1285f250 P 05c90d307b174f4381f2756f544d8072: Deleting consensus metadata
I20260809 04:44:14.816558 26859 ts_tablet_manager.cc:1918] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:14.816598 26306 catalog_manager.cc:4841] TS 05c90d307b174f4381f2756f544d8072 (127.5.222.195:36803): tablet deab857fc98d4367a140dbab1285f250 (table test-table [id=a7361856344a419aadf2de9d604c1dac]) successfully deleted
I20260809 04:44:14.816674 26859 log.cc:1196] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/deab857fc98d4367a140dbab1285f250
I20260809 04:44:14.817073 26859 ts_tablet_manager.cc:1939] T deab857fc98d4367a140dbab1285f250 P 7d25059fb52d41849dd8cacb84c88974: Deleting consensus metadata
I20260809 04:44:14.818307 26307 catalog_manager.cc:4841] TS 7d25059fb52d41849dd8cacb84c88974 (127.5.222.194:43987): tablet deab857fc98d4367a140dbab1285f250 (table test-table [id=a7361856344a419aadf2de9d604c1dac]) successfully deleted
W20260809 04:44:14.819159 26308 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:42539 (error 108)
I20260809 04:44:14.819447 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 26534
I20260809 04:44:14.821882 26827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.821506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:42539 (local address 127.0.0.1:60110)
0809 04:44:14.821701 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:14.821786 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:42539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260809 04:44:14.822132 26308 catalog_manager.cc:4584] TS 9731309bbf1a4886898350731686e934 (127.5.222.193:42539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet deab857fc98d4367a140dbab1285f250: Network error: Client connection negotiation failed: client connection to 127.5.222.193:42539: connect: Connection refused (error 111)
I20260809 04:44:14.832276 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 26675
I20260809 04:44:14.839709 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 26288
2026-08-09T04:44:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2473 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T04:44:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:14Z Disabled control of system clock
I20260809 04:44:14.872187 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:40903
--webserver_interface=127.5.222.254
--webserver_port=0
--builtin_ntp_servers=127.5.222.212:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:40903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:44:14.981664 26865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:14.981935 26865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:14.981971 26865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:14.985948 26865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:14.986009 26865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:14.986025 26865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:14.986042 26865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:14.986057 26865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:14.990875 26865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:40903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:40903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:14.991914 26865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:14.993250 26865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:15.001652 26870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:15.001686 26865 server_base.cc:1034] running on GCE node
W20260809 04:44:15.001642 26871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:15.001658 26873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:15.002435 26865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:15.003598 26865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:15.004812 26865 hybrid_clock.cc:648] HybridClock initialized: now 1786250655004794 us; error 31 us; skew 500 ppm
I20260809 04:44:15.006910 26865 webserver.cc:460] Webserver started at http://127.5.222.254:36693/ using document root <none> and password file <none>
I20260809 04:44:15.007529 26865 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:15.007596 26865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:15.007862 26865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:15.010136 26865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "50ef1d4b2afa42619d2b1217a8789931"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-vxj2"
server_key: "4012cd257ec09c7faf55a6ee519fdaa5"
server_key_iv: "838b625168b64456ab7f1906c28c0c0c"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.010766 26865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "50ef1d4b2afa42619d2b1217a8789931"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-vxj2"
server_key: "4012cd257ec09c7faf55a6ee519fdaa5"
server_key_iv: "838b625168b64456ab7f1906c28c0c0c"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.014595 26865 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:44:15.017287 26879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.018875 26865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:15.019021 26865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "50ef1d4b2afa42619d2b1217a8789931"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-vxj2"
server_key: "4012cd257ec09c7faf55a6ee519fdaa5"
server_key_iv: "838b625168b64456ab7f1906c28c0c0c"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.019140 26865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:15.040850 26865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:15.041633 26865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:15.041858 26865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:15.049417 26865 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:40903
I20260809 04:44:15.049429 26937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:40903 every 8 connection(s)
I20260809 04:44:15.050390 26865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:15.053541 26938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:15.058310 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 26865
I20260809 04:44:15.058467 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:15.058852 6011 external_mini_cluster.cc:1428] Setting key 6a38e70f54eab655857f8cc47bb5f08f
I20260809 04:44:15.059144 26938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931: Bootstrap starting.
I20260809 04:44:15.061652 26938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:15.062381 26938 log.cc:824] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:15.064424 26938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931: No bootstrap required, opened a new log
I20260809 04:44:15.068670 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.060249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44996 (local address 127.5.222.254:40903)
0809 04:44:15.060875 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.060891 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.060927 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:44:15.061614 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.061632 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.062202 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.062605 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.063162 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.064156 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.065399 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.065897 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.066032 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.066898 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.066902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.066914 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:15.067277 (+ 363us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.067330 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:15.067349 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.067425 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.067613 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.067932 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":139,"threads_started":1}
I20260809 04:44:15.068572 26938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ef1d4b2afa42619d2b1217a8789931" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40903 } }
I20260809 04:44:15.069010 26938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:15.069105 26938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ef1d4b2afa42619d2b1217a8789931, State: Initialized, Role: FOLLOWER
I20260809 04:44:15.069561 26938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ef1d4b2afa42619d2b1217a8789931" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40903 } }
I20260809 04:44:15.069728 26938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:15.069849 26938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:15.069957 26938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:15.071020 26938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ef1d4b2afa42619d2b1217a8789931" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40903 } }
I20260809 04:44:15.071362 26938 leader_election.cc:302] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50ef1d4b2afa42619d2b1217a8789931; no voters:
I20260809 04:44:15.071651 26938 leader_election.cc:288] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:15.071879 26944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:15.072151 26944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [term 1 LEADER]: Becoming Leader. State: Replica: 50ef1d4b2afa42619d2b1217a8789931, State: Running, Role: LEADER
I20260809 04:44:15.072552 26944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ef1d4b2afa42619d2b1217a8789931" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40903 } }
I20260809 04:44:15.072990 26938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:15.074442 26945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50ef1d4b2afa42619d2b1217a8789931" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ef1d4b2afa42619d2b1217a8789931" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40903 } } }
I20260809 04:44:15.074595 26945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:15.074501 26946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50ef1d4b2afa42619d2b1217a8789931. Latest consensus state: current_term: 1 leader_uuid: "50ef1d4b2afa42619d2b1217a8789931" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ef1d4b2afa42619d2b1217a8789931" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40903 } } }
I20260809 04:44:15.074733 26946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:15.075721 26956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:15.078655 26956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:15.084165 26956 catalog_manager.cc:1347] Generated new cluster ID: 27ea10af8280477398e7ba7423c95e62
I20260809 04:44:15.084267 26956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:15.093218 26956 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:15.094357 26956 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:15.100871 26956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931: Generated new TSK 0
I20260809 04:44:15.101526 26956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:15.106454 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40903
--builtin_ntp_servers=127.5.222.212:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:44:15.221276 26963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:15.221552 26963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:15.221634 26963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:15.225934 26963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:15.226017 26963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:15.226109 26963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:44:15.230834 26963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:15.231930 26963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:15.233441 26963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:15.243170 26968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:15.243237 26971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:15.243180 26969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:15.243386 26963 server_base.cc:1034] running on GCE node
I20260809 04:44:15.243901 26963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:15.244627 26963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:15.245800 26963 hybrid_clock.cc:648] HybridClock initialized: now 1786250655245778 us; error 35 us; skew 500 ppm
I20260809 04:44:15.247831 26963 webserver.cc:460] Webserver started at http://127.5.222.193:44037/ using document root <none> and password file <none>
I20260809 04:44:15.248423 26963 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:15.248486 26963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:15.248710 26963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:15.250936 26963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "d1c73d460e1746feb947cd1674f7f5b1"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-vxj2"
server_key: "42765f9a83ad09baf3ee6bfc692cffb7"
server_key_iv: "e0668e571c4fcd4066d13bb6899fef65"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.251500 26963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "d1c73d460e1746feb947cd1674f7f5b1"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-vxj2"
server_key: "42765f9a83ad09baf3ee6bfc692cffb7"
server_key_iv: "e0668e571c4fcd4066d13bb6899fef65"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.255342 26963 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 04:44:15.258015 26978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.259294 26963 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:44:15.259454 26963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "d1c73d460e1746feb947cd1674f7f5b1"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-vxj2"
server_key: "42765f9a83ad09baf3ee6bfc692cffb7"
server_key_iv: "e0668e571c4fcd4066d13bb6899fef65"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.259577 26963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:15.274286 26963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:15.274998 26963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:15.275204 26963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:15.275815 26963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:15.276779 26963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:15.276850 26963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.276924 26963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:15.276970 26963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.286320 26963 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:40083
I20260809 04:44:15.286346 27097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:40083 every 8 connection(s)
I20260809 04:44:15.287189 26963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:15.293606 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 26963
I20260809 04:44:15.293741 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:15.294063 6011 external_mini_cluster.cc:1428] Setting key 685c75b0a9872390d9c441d64306d59d
I20260809 04:44:15.296020 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40903
--builtin_ntp_servers=127.5.222.212:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:44:15.296578 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.288912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:54307 (local address 127.5.222.254:40903)
0809 04:44:15.289057 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.289061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.290131 (+ 1070us) server_negotiation.cc:407] Connection header received
0809 04:44:15.291532 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.291536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.291596 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.291698 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.292567 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.293130 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.294503 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.294742 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.294860 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.295942 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.295946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.295948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:15.295961 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.295990 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:15.295998 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.296078 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.296301 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.296376 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:44:15.297343 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.289304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40903 (local address 127.5.222.193:54307)
0809 04:44:15.289979 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:15.290020 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:15.291310 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:15.291875 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:15.291881 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:15.291958 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:15.292423 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.292435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.293311 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.293314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.294375 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.294386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.294895 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.294899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.295060 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.295683 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:15.295685 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:15.295722 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:15.295787 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:15.296082 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:15.296093 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:15.296100 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 04:44:15.296281 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 04:44:15.296488 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
I20260809 04:44:15.298621 27098 heartbeater.cc:347] Connected to a master server at 127.5.222.254:40903
I20260809 04:44:15.298913 27098 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:15.299499 27098 heartbeater.cc:511] Master 127.5.222.254:40903 requested a full tablet report, sending...
I20260809 04:44:15.300958 26899 ts_manager.cc:194] Registered new tserver with Master: d1c73d460e1746feb947cd1674f7f5b1 (127.5.222.193:40083)
I20260809 04:44:15.302153 26899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.222.193:54307
W20260809 04:44:15.412022 27101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:15.412312 27101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:15.412384 27101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:15.416371 27101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:15.416424 27101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:15.416476 27101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:44:15.421047 27101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:15.422055 27101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:15.423584 27101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:15.433140 27111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:15.433128 27108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:15.433362 27101 server_base.cc:1034] running on GCE node
W20260809 04:44:15.433126 27109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:15.433837 27101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:15.434581 27101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:15.435745 27101 hybrid_clock.cc:648] HybridClock initialized: now 1786250655435733 us; error 25 us; skew 500 ppm
I20260809 04:44:15.437654 27101 webserver.cc:460] Webserver started at http://127.5.222.194:40251/ using document root <none> and password file <none>
I20260809 04:44:15.438227 27101 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:15.438290 27101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:15.438566 27101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:15.440665 27101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "72e84d5d2d1b40ce8092938080a096cc"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-vxj2"
server_key: "51eafa3f8f5986e26fa9c84ccaf837e3"
server_key_iv: "391277990ad70a43b45a21fd5c43b09e"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.441283 27101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "72e84d5d2d1b40ce8092938080a096cc"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-vxj2"
server_key: "51eafa3f8f5986e26fa9c84ccaf837e3"
server_key_iv: "391277990ad70a43b45a21fd5c43b09e"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.444929 27101 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:15.447427 27120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.448693 27101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:44:15.448796 27101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "72e84d5d2d1b40ce8092938080a096cc"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-vxj2"
server_key: "51eafa3f8f5986e26fa9c84ccaf837e3"
server_key_iv: "391277990ad70a43b45a21fd5c43b09e"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.448935 27101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:15.457983 27101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:15.458606 27101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:15.458797 27101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:15.459354 27101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:15.460280 27101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:15.460328 27101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.460394 27101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:15.460427 27101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.469465 27101 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:42377
I20260809 04:44:15.469491 27241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:42377 every 8 connection(s)
I20260809 04:44:15.470319 27101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:15.473511 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 27101
I20260809 04:44:15.473658 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:15.474079 6011 external_mini_cluster.cc:1428] Setting key 7bc0d015a573acc84583e266e0d21dc9
I20260809 04:44:15.476365 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40903
--builtin_ntp_servers=127.5.222.212:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:44:15.481087 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.472027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:51669 (local address 127.5.222.254:40903)
0809 04:44:15.472164 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.472168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.473383 (+ 1215us) server_negotiation.cc:407] Connection header received
0809 04:44:15.475277 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.475281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.475344 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.475464 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.476680 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.477270 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.478737 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.479145 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.479272 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.480437 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.480443 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.480448 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:15.480467 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.480496 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:15.480510 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.480610 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.480780 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.480862 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:44:15.481969 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.472352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40903 (local address 127.5.222.194:51669)
0809 04:44:15.473117 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:15.473235 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:15.475037 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:15.475643 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:15.475652 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:15.475755 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:15.476458 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.476484 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.477410 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.477417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.478595 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.478605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.479305 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.479308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.479453 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.480149 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:15.480150 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:15.480190 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:15.480261 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:15.480615 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:15.480623 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:15.480628 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:44:15.480740 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 04:44:15.480958 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":128,"threads_started":1}
I20260809 04:44:15.483192 27242 heartbeater.cc:347] Connected to a master server at 127.5.222.254:40903
I20260809 04:44:15.483455 27242 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:15.484148 27242 heartbeater.cc:511] Master 127.5.222.254:40903 requested a full tablet report, sending...
I20260809 04:44:15.485409 26899 ts_manager.cc:194] Registered new tserver with Master: 72e84d5d2d1b40ce8092938080a096cc (127.5.222.194:42377)
I20260809 04:44:15.486069 26899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.222.194:51669
W20260809 04:44:15.590519 27245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:15.590778 27245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:15.590847 27245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:15.594882 27245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:15.594939 27245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:15.594993 27245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:44:15.599963 27245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:15.601275 27245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:15.602769 27245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:15.612787 27252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:15.612957 27245 server_base.cc:1034] running on GCE node
W20260809 04:44:15.612797 27254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:15.612859 27251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:15.613494 27245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:15.614187 27245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:15.615353 27245 hybrid_clock.cc:648] HybridClock initialized: now 1786250655615340 us; error 25 us; skew 500 ppm
I20260809 04:44:15.617209 27245 webserver.cc:460] Webserver started at http://127.5.222.195:39893/ using document root <none> and password file <none>
I20260809 04:44:15.617823 27245 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:15.617890 27245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:15.618127 27245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:15.620200 27245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "b8653773619c42908bb9d8591e652783"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-vxj2"
server_key: "6941cdc64e4fe4f71d62078b051dd9bd"
server_key_iv: "15f9b8ea782bae19611059753a24e227"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.620784 27245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "b8653773619c42908bb9d8591e652783"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-vxj2"
server_key: "6941cdc64e4fe4f71d62078b051dd9bd"
server_key_iv: "15f9b8ea782bae19611059753a24e227"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.624711 27245 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 04:44:15.627171 27262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.628372 27245 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:44:15.628515 27245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "b8653773619c42908bb9d8591e652783"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-vxj2"
server_key: "6941cdc64e4fe4f71d62078b051dd9bd"
server_key_iv: "15f9b8ea782bae19611059753a24e227"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.628628 27245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:15.654472 27245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:15.655217 27245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:15.655428 27245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:15.656047 27245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:15.657006 27245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:15.657079 27245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.657166 27245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:15.657212 27245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.666957 27245 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:42129
I20260809 04:44:15.666992 27382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:42129 every 8 connection(s)
I20260809 04:44:15.667877 27245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:15.673671 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 27245
I20260809 04:44:15.673791 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:15.674132 6011 external_mini_cluster.cc:1428] Setting key 436be7ec6465cedd37482da12f37f397
I20260809 04:44:15.678084 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.669869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:55851 (local address 127.5.222.254:40903)
0809 04:44:15.670028 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.670031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.671119 (+ 1088us) server_negotiation.cc:407] Connection header received
0809 04:44:15.672538 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.672541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.672600 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.672708 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.673832 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.674628 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.676202 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.676495 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.676578 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.677719 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.677722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.677724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:15.677735 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.677751 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:15.677758 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.677815 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.677925 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.677974 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:44:15.678905 27385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.670262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40903 (local address 127.5.222.195:55851)
0809 04:44:15.670963 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:15.671004 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:15.672323 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:15.672917 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:15.672924 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:15.673012 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:15.673660 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.673683 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.674805 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.674809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.676026 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.676036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.676630 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.676633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.676775 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.677489 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:15.677491 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:15.677529 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:15.677589 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:15.677814 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:15.677818 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:15.677822 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:44:15.677906 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:44:15.678109 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":123,"threads_started":1}
I20260809 04:44:15.679956 27383 heartbeater.cc:347] Connected to a master server at 127.5.222.254:40903
I20260809 04:44:15.680179 27383 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:15.680683 27383 heartbeater.cc:511] Master 127.5.222.254:40903 requested a full tablet report, sending...
I20260809 04:44:15.681732 26899 ts_manager.cc:194] Registered new tserver with Master: b8653773619c42908bb9d8591e652783 (127.5.222.195:42129)
I20260809 04:44:15.682286 26899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.222.195:55851
I20260809 04:44:15.688019 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:44:15.693552 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.690020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.5.222.254:40903)
0809 04:44:15.690141 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.690145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.690309 (+ 164us) server_negotiation.cc:407] Connection header received
0809 04:44:15.690405 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.690407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.690456 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.690520 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.690751 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.691239 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.692088 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.692295 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.692366 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.693119 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.693122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.693124 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:15.693136 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.693201 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:15.693220 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.693324 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.693409 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.693451 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:44:15.696204 26899 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:15.699254 26899 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:15.714604 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.709409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56802 (local address 127.5.222.193:40083)
0809 04:44:15.709815 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.709828 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.710031 (+ 203us) server_negotiation.cc:407] Connection header received
0809 04:44:15.710150 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.710154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.710335 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.710502 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:15.710995 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.712152 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.713349 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.713906 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.714045 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.714252 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.714344 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.714450 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":107,"threads_started":1}
I20260809 04:44:15.714603 27395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.708381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:40083 (local address 127.0.0.1:56802)
0809 04:44:15.709920 (+ 1539us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:15.709930 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:15.710038 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:15.710652 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:15.710655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:15.710687 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:15.710850 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.710859 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.712256 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.712259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.713212 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.713226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.714044 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.714047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.714159 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.714195 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 04:44:15.714306 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 04:44:15.714432 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1487,"mutex_wait_us":17,"spinlock_wait_cycles":5888,"thread_start_us":110,"threads_started":1}
I20260809 04:44:15.715808 27396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.708288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:42377 (local address 127.0.0.1:59218)
0809 04:44:15.709477 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:15.709531 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:15.709736 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:15.710337 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:15.710343 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:15.710367 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:15.710527 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.710537 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.712076 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.712079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.713912 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.713924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.715307 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.715313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.715445 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.715469 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:44:15.715558 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 04:44:15.715638 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"mutex_wait_us":296,"thread_start_us":97,"threads_started":1}
I20260809 04:44:15.716259 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.708414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59218 (local address 127.5.222.194:42377)
0809 04:44:15.709021 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.709040 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.709624 (+ 584us) server_negotiation.cc:407] Connection header received
0809 04:44:15.709900 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.709907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.710151 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.710348 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:15.710663 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.711935 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.714071 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.714930 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.715048 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.715229 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.716004 (+ 775us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.716103 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":117,"threads_started":1}
I20260809 04:44:15.716302 27399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.709266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38848 (local address 127.5.222.195:42129)
0809 04:44:15.709722 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.709747 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.710156 (+ 409us) server_negotiation.cc:407] Connection header received
0809 04:44:15.710231 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.710236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.710379 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.710509 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:15.711023 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.711834 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.714071 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.715248 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.715398 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.715611 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.716076 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.716161 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":136,"threads_started":1}
I20260809 04:44:15.716687 27398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.709043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:42129 (local address 127.0.0.1:38848)
0809 04:44:15.709972 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:15.709989 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:15.710088 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:15.710652 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:15.710655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:15.710685 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:15.710860 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.710866 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.711970 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.711973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.713912 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.713924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.715678 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.715681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.715770 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.715792 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:44:15.715872 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 04:44:15.716539 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"mutex_wait_us":368,"spinlock_wait_cycles":7040,"thread_start_us":126,"threads_started":1}
I20260809 04:44:15.718350 27031 tablet_service.cc:1467] Processing CreateTablet for tablet 477c6ca53900443da33d69246c83170e (DEFAULT_TABLE table=test-table [id=e0bee1a47bae4e19917ecfb1a18febe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:15.719544 27031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 477c6ca53900443da33d69246c83170e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:15.719373 27169 tablet_service.cc:1467] Processing CreateTablet for tablet 477c6ca53900443da33d69246c83170e (DEFAULT_TABLE table=test-table [id=e0bee1a47bae4e19917ecfb1a18febe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:15.720307 27169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 477c6ca53900443da33d69246c83170e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:15.720703 27314 tablet_service.cc:1467] Processing CreateTablet for tablet 477c6ca53900443da33d69246c83170e (DEFAULT_TABLE table=test-table [id=e0bee1a47bae4e19917ecfb1a18febe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:15.721680 27314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 477c6ca53900443da33d69246c83170e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:15.726581 27401 tablet_bootstrap.cc:492] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc: Bootstrap starting.
I20260809 04:44:15.728861 27401 tablet_bootstrap.cc:654] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:15.728997 27403 tablet_bootstrap.cc:492] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783: Bootstrap starting.
I20260809 04:44:15.729460 27402 tablet_bootstrap.cc:492] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1: Bootstrap starting.
I20260809 04:44:15.730127 27401 log.cc:824] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:15.731383 27403 tablet_bootstrap.cc:654] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:15.731624 27402 tablet_bootstrap.cc:654] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:15.732170 27403 log.cc:824] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:15.732327 27401 tablet_bootstrap.cc:492] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc: No bootstrap required, opened a new log
I20260809 04:44:15.732421 27402 log.cc:824] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:15.732533 27401 ts_tablet_manager.cc:1397] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:44:15.734328 27403 tablet_bootstrap.cc:492] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783: No bootstrap required, opened a new log
I20260809 04:44:15.734452 27402 tablet_bootstrap.cc:492] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1: No bootstrap required, opened a new log
I20260809 04:44:15.734513 27403 ts_tablet_manager.cc:1397] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:44:15.734648 27402 ts_tablet_manager.cc:1397] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:44:15.737991 27402 raft_consensus.cc:348] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e84d5d2d1b40ce8092938080a096cc" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 42377 } } peers { permanent_uuid: "d1c73d460e1746feb947cd1674f7f5b1" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 40083 } } peers { permanent_uuid: "b8653773619c42908bb9d8591e652783" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42129 } }
I20260809 04:44:15.738037 27401 raft_consensus.cc:348] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e84d5d2d1b40ce8092938080a096cc" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 42377 } } peers { permanent_uuid: "d1c73d460e1746feb947cd1674f7f5b1" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 40083 } } peers { permanent_uuid: "b8653773619c42908bb9d8591e652783" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42129 } }
I20260809 04:44:15.738257 27402 raft_consensus.cc:374] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:15.738291 27401 raft_consensus.cc:374] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:15.738309 27402 raft_consensus.cc:729] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c73d460e1746feb947cd1674f7f5b1, State: Initialized, Role: FOLLOWER
I20260809 04:44:15.738343 27401 raft_consensus.cc:729] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72e84d5d2d1b40ce8092938080a096cc, State: Initialized, Role: FOLLOWER
I20260809 04:44:15.738583 27403 raft_consensus.cc:348] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e84d5d2d1b40ce8092938080a096cc" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 42377 } } peers { permanent_uuid: "d1c73d460e1746feb947cd1674f7f5b1" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 40083 } } peers { permanent_uuid: "b8653773619c42908bb9d8591e652783" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42129 } }
I20260809 04:44:15.738811 27403 raft_consensus.cc:374] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:15.738858 27403 raft_consensus.cc:729] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8653773619c42908bb9d8591e652783, State: Initialized, Role: FOLLOWER
I20260809 04:44:15.738799 27401 consensus_queue.cc:260] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e84d5d2d1b40ce8092938080a096cc" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 42377 } } peers { permanent_uuid: "d1c73d460e1746feb947cd1674f7f5b1" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 40083 } } peers { permanent_uuid: "b8653773619c42908bb9d8591e652783" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42129 } }
I20260809 04:44:15.738816 27402 consensus_queue.cc:260] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e84d5d2d1b40ce8092938080a096cc" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 42377 } } peers { permanent_uuid: "d1c73d460e1746feb947cd1674f7f5b1" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 40083 } } peers { permanent_uuid: "b8653773619c42908bb9d8591e652783" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42129 } }
I20260809 04:44:15.739246 27403 consensus_queue.cc:260] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e84d5d2d1b40ce8092938080a096cc" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 42377 } } peers { permanent_uuid: "d1c73d460e1746feb947cd1674f7f5b1" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 40083 } } peers { permanent_uuid: "b8653773619c42908bb9d8591e652783" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42129 } }
I20260809 04:44:15.739583 27242 heartbeater.cc:503] Master 127.5.222.254:40903 was elected leader, sending a full tablet report...
I20260809 04:44:15.739858 27401 ts_tablet_manager.cc:1428] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 04:44:15.739863 27402 ts_tablet_manager.cc:1428] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 04:44:15.740051 27403 ts_tablet_manager.cc:1428] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260809 04:44:15.740235 27098 heartbeater.cc:503] Master 127.5.222.254:40903 was elected leader, sending a full tablet report...
I20260809 04:44:15.740358 27383 heartbeater.cc:503] Master 127.5.222.254:40903 was elected leader, sending a full tablet report...
W20260809 04:44:15.788519 27099 tablet.cc:2369] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:44:15.918934 27384 tablet.cc:2369] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:44:15.971544 27243 tablet.cc:2369] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:15.987983 27408 raft_consensus.cc:482] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:15.988212 27408 raft_consensus.cc:504] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e84d5d2d1b40ce8092938080a096cc" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 42377 } } peers { permanent_uuid: "d1c73d460e1746feb947cd1674f7f5b1" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 40083 } } peers { permanent_uuid: "b8653773619c42908bb9d8591e652783" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42129 } }
I20260809 04:44:15.989442 27408 leader_election.cc:288] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c73d460e1746feb947cd1674f7f5b1 (127.5.222.193:40083), b8653773619c42908bb9d8591e652783 (127.5.222.195:42129)
I20260809 04:44:15.993281 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.989836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:33187 (local address 127.5.222.193:40083)
0809 04:44:15.989986 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.989990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.990152 (+ 162us) server_negotiation.cc:407] Connection header received
0809 04:44:15.990296 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.990300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.990356 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.990453 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:15.990737 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.991244 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.992323 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.992709 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.992846 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.992898 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.993071 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.993136 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:44:15.993301 27411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.989788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:40083 (local address 127.5.222.194:33187)
0809 04:44:15.990036 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:15.990051 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:15.990172 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:15.990478 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:15.990482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:15.990502 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:15.990611 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.990617 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.991365 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.991370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.992189 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.992197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.992837 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.992841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.992935 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.992959 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:44:15.993052 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 04:44:15.993120 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260809 04:44:15.993801 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.990180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:42129 (local address 127.5.222.194:51107)
0809 04:44:15.990538 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:15.990550 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:15.990673 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:15.990919 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:15.990922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:15.990938 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:15.991063 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.991068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.991818 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.991824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.992597 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.992606 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.993320 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.993323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.993445 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.993476 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 04:44:15.993593 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 04:44:15.993654 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":82,"threads_started":1}
I20260809 04:44:15.993845 27399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.990343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:51107 (local address 127.5.222.195:42129)
0809 04:44:15.990495 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.990500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.990629 (+ 129us) server_negotiation.cc:407] Connection header received
0809 04:44:15.990764 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.990766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.990817 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.990899 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:15.991171 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.991687 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.992727 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.993139 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.993332 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.993391 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.993647 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.993733 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:44:15.993944 27052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477c6ca53900443da33d69246c83170e" candidate_uuid: "72e84d5d2d1b40ce8092938080a096cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c73d460e1746feb947cd1674f7f5b1" is_pre_election: true
I20260809 04:44:15.994264 27052 raft_consensus.cc:2454] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72e84d5d2d1b40ce8092938080a096cc in term 0.
I20260809 04:44:15.994400 27334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477c6ca53900443da33d69246c83170e" candidate_uuid: "72e84d5d2d1b40ce8092938080a096cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8653773619c42908bb9d8591e652783" is_pre_election: true
I20260809 04:44:15.994689 27334 raft_consensus.cc:2454] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72e84d5d2d1b40ce8092938080a096cc in term 0.
I20260809 04:44:15.994745 27122 leader_election.cc:302] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72e84d5d2d1b40ce8092938080a096cc, d1c73d460e1746feb947cd1674f7f5b1; no voters:
I20260809 04:44:15.995075 27408 raft_consensus.cc:2781] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:15.995169 27408 raft_consensus.cc:482] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:15.995214 27408 raft_consensus.cc:3037] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:15.996261 27408 raft_consensus.cc:504] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e84d5d2d1b40ce8092938080a096cc" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 42377 } } peers { permanent_uuid: "d1c73d460e1746feb947cd1674f7f5b1" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 40083 } } peers { permanent_uuid: "b8653773619c42908bb9d8591e652783" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42129 } }
I20260809 04:44:15.996603 27408 leader_election.cc:288] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [CANDIDATE]: Term 1 election: Requested vote from peers d1c73d460e1746feb947cd1674f7f5b1 (127.5.222.193:40083), b8653773619c42908bb9d8591e652783 (127.5.222.195:42129)
I20260809 04:44:15.996985 27052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477c6ca53900443da33d69246c83170e" candidate_uuid: "72e84d5d2d1b40ce8092938080a096cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c73d460e1746feb947cd1674f7f5b1"
I20260809 04:44:15.997007 27334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477c6ca53900443da33d69246c83170e" candidate_uuid: "72e84d5d2d1b40ce8092938080a096cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8653773619c42908bb9d8591e652783"
I20260809 04:44:15.997138 27052 raft_consensus.cc:3037] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:15.997201 27334 raft_consensus.cc:3037] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:15.998200 27052 raft_consensus.cc:2454] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72e84d5d2d1b40ce8092938080a096cc in term 1.
I20260809 04:44:15.998417 27334 raft_consensus.cc:2454] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72e84d5d2d1b40ce8092938080a096cc in term 1.
I20260809 04:44:15.998541 27122 leader_election.cc:302] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72e84d5d2d1b40ce8092938080a096cc, d1c73d460e1746feb947cd1674f7f5b1; no voters:
I20260809 04:44:15.998744 27408 raft_consensus.cc:2781] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:15.998971 27408 raft_consensus.cc:686] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 1 LEADER]: Becoming Leader. State: Replica: 72e84d5d2d1b40ce8092938080a096cc, State: Running, Role: LEADER
I20260809 04:44:15.999328 27408 consensus_queue.cc:237] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e84d5d2d1b40ce8092938080a096cc" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 42377 } } peers { permanent_uuid: "d1c73d460e1746feb947cd1674f7f5b1" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 40083 } } peers { permanent_uuid: "b8653773619c42908bb9d8591e652783" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42129 } }
I20260809 04:44:16.002570 26898 catalog_manager.cc:5495] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 72e84d5d2d1b40ce8092938080a096cc (127.5.222.194). New cstate: current_term: 1 leader_uuid: "72e84d5d2d1b40ce8092938080a096cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e84d5d2d1b40ce8092938080a096cc" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1c73d460e1746feb947cd1674f7f5b1" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 40083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8653773619c42908bb9d8591e652783" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42129 } health_report { overall_health: UNKNOWN } } }
I20260809 04:44:16.084416 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:16.080794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59226 (local address 127.5.222.194:42377)
0809 04:44:16.080952 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:16.080956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:16.080967 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:44:16.081032 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:16.081034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:16.081087 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:16.081238 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:16.081507 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:16.082015 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:16.082894 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:16.083121 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:16.083235 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:16.083561 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:16.083959 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:16.084120 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:16.084245 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 04:44:16.084285 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:44:16.090435 27334 raft_consensus.cc:1261] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783 [term 1 FOLLOWER]: Refusing update from remote peer 72e84d5d2d1b40ce8092938080a096cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:16.090453 27052 raft_consensus.cc:1261] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1 [term 1 FOLLOWER]: Refusing update from remote peer 72e84d5d2d1b40ce8092938080a096cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:16.091197 27414 consensus_queue.cc:1035] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c73d460e1746feb947cd1674f7f5b1" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:16.091372 27408 consensus_queue.cc:1035] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8653773619c42908bb9d8591e652783" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 42129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:16.098372 27417 mvcc.cc:205] Tried to move back new op lower bound from 7316482687338946560 to 7316482686975725568. Current Snapshot: MvccSnapshot[applied={T|T < 7316482687338946560}]
I20260809 04:44:16.100692 27419 mvcc.cc:205] Tried to move back new op lower bound from 7316482687338946560 to 7316482686975725568. Current Snapshot: MvccSnapshot[applied={T|T < 7316482687338946560}]
I20260809 04:44:16.107290 26895 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:16.107470 26895 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:16.110116 26895 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50ef1d4b2afa42619d2b1217a8789931: Sending DeleteTablet for 3 replicas of tablet 477c6ca53900443da33d69246c83170e
I20260809 04:44:16.110896 27169 tablet_service.cc:1514] Processing DeleteTablet for tablet 477c6ca53900443da33d69246c83170e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:16 UTC) from {username='slave'} at 127.0.0.1:59218
I20260809 04:44:16.111228 27031 tablet_service.cc:1514] Processing DeleteTablet for tablet 477c6ca53900443da33d69246c83170e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:16 UTC) from {username='slave'} at 127.0.0.1:56802
I20260809 04:44:16.111279 27314 tablet_service.cc:1514] Processing DeleteTablet for tablet 477c6ca53900443da33d69246c83170e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:16 UTC) from {username='slave'} at 127.0.0.1:38848
I20260809 04:44:16.111871 27427 tablet_replica.cc:330] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1: stopping tablet replica
I20260809 04:44:16.111876 27430 tablet_replica.cc:330] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783: stopping tablet replica
I20260809 04:44:16.112120 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 26963
I20260809 04:44:16.112210 27427 raft_consensus.cc:2229] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:16.112372 27430 raft_consensus.cc:2229] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:16.112557 27427 raft_consensus.cc:2258] T 477c6ca53900443da33d69246c83170e P d1c73d460e1746feb947cd1674f7f5b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:16.112782 27430 raft_consensus.cc:2258] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:16.113226 27426 tablet_replica.cc:330] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc: stopping tablet replica
I20260809 04:44:16.113551 27426 raft_consensus.cc:2229] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:16.113958 27426 raft_consensus.cc:2258] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:16.118312 27426 ts_tablet_manager.cc:1905] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:16.118319 27430 ts_tablet_manager.cc:1905] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:16.121209 27426 ts_tablet_manager.cc:1918] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:16.121315 27426 log.cc:1196] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/477c6ca53900443da33d69246c83170e
I20260809 04:44:16.121621 27426 ts_tablet_manager.cc:1939] T 477c6ca53900443da33d69246c83170e P 72e84d5d2d1b40ce8092938080a096cc: Deleting consensus metadata
I20260809 04:44:16.122058 27430 ts_tablet_manager.cc:1918] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:16.122179 27430 log.cc:1196] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/477c6ca53900443da33d69246c83170e
I20260809 04:44:16.122560 27430 ts_tablet_manager.cc:1939] T 477c6ca53900443da33d69246c83170e P b8653773619c42908bb9d8591e652783: Deleting consensus metadata
W20260809 04:44:16.122632 26881 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:40083 (error 108)
I20260809 04:44:16.122834 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 27101
I20260809 04:44:16.127226 26881 catalog_manager.cc:4841] TS b8653773619c42908bb9d8591e652783 (127.5.222.195:42129): tablet 477c6ca53900443da33d69246c83170e (table test-table [id=e0bee1a47bae4e19917ecfb1a18febe6]) successfully deleted
I20260809 04:44:16.127344 27398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:16.127023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:40083 (local address 127.0.0.1:56804)
0809 04:44:16.127160 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:16.127247 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:40083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 04:44:16.127471 26881 catalog_manager.cc:4584] TS d1c73d460e1746feb947cd1674f7f5b1 (127.5.222.193:40083): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 477c6ca53900443da33d69246c83170e: Network error: Client connection negotiation failed: client connection to 127.5.222.193:40083: connect: Connection refused (error 111)
I20260809 04:44:16.130669 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 27245
I20260809 04:44:16.131805 27398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:16.130855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:42377 (local address 127.0.0.1:59238)
0809 04:44:16.131586 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:16.131651 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.194:42377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":624}
I20260809 04:44:16.141996 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 26865
2026-08-09T04:44:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1301 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:16 dist-test-slave-vxj2 krb5kdc[27434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:16 dist-test-slave-vxj2 krb5kdc[27434](info): set up 2 sockets
Aug 09 04:44:16 dist-test-slave-vxj2 krb5kdc[27434](info): commencing operation
krb5kdc: starting...
W20260809 04:44:17.247653 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:17 dist-test-slave-vxj2 krb5kdc[27434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:44:17.371856 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:40545
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:40545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:17.484562 27448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:17.484856 27448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:17.484932 27448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:17.489492 27448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:17.489583 27448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:17.489624 27448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:17.489682 27448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:17.489719 27448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:17.495034 27448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:40545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:40545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:17.496054 27448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:17.497059 27448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:17.505581 27453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:17.505568 27456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:17.505565 27454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:17.505893 27448 server_base.cc:1034] running on GCE node
I20260809 04:44:17.506435 27448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:17.507575 27448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:17.508736 27448 hybrid_clock.cc:648] HybridClock initialized: now 1786250657508718 us; error 30 us; skew 500 ppm
Aug 09 04:44:17 dist-test-slave-vxj2 krb5kdc[27434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:17.511504 27448 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:44:17.512356 27448 webserver.cc:460] Webserver started at http://127.5.222.254:42919/ using document root <none> and password file <none>
I20260809 04:44:17.512854 27448 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:17.512920 27448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:17.513080 27448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:17.515076 27448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "8fe1cf6cb81f42b391b5efba511f3683"
format_stamp: "Formatted at 2026-08-09 04:44:17 on dist-test-slave-vxj2"
server_key: "087b33a816fcfd81cf83099290267f8f"
server_key_iv: "af9d8d0bd02bb420609c704848711e80"
server_key_version: "encryptionkey@0"
I20260809 04:44:17.515580 27448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "8fe1cf6cb81f42b391b5efba511f3683"
format_stamp: "Formatted at 2026-08-09 04:44:17 on dist-test-slave-vxj2"
server_key: "087b33a816fcfd81cf83099290267f8f"
server_key_iv: "af9d8d0bd02bb420609c704848711e80"
server_key_version: "encryptionkey@0"
I20260809 04:44:17.519320 27448 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:17.521642 27463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:17.522826 27448 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:17.522977 27448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "8fe1cf6cb81f42b391b5efba511f3683"
format_stamp: "Formatted at 2026-08-09 04:44:17 on dist-test-slave-vxj2"
server_key: "087b33a816fcfd81cf83099290267f8f"
server_key_iv: "af9d8d0bd02bb420609c704848711e80"
server_key_version: "encryptionkey@0"
I20260809 04:44:17.523088 27448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:17.541263 27448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:17.544112 27448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:17.544330 27448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:17.551946 27448 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:40545
I20260809 04:44:17.551946 27518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:40545 every 8 connection(s)
I20260809 04:44:17.552942 27448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:17.555701 27519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:17.558374 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 27448
I20260809 04:44:17.558530 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:17.558808 6011 external_mini_cluster.cc:1428] Setting key 225119823cd6d7abe5a923b8ba0c55a5
I20260809 04:44:17.561875 27519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683: Bootstrap starting.
I20260809 04:44:17.564038 27519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:17.564771 27519 log.cc:824] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:17.566854 27519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683: No bootstrap required, opened a new log
Aug 09 04:44:17 dist-test-slave-vxj2 krb5kdc[27434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:17.570137 27519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe1cf6cb81f42b391b5efba511f3683" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40545 } }
I20260809 04:44:17.570408 27519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:17.570495 27519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe1cf6cb81f42b391b5efba511f3683, State: Initialized, Role: FOLLOWER
I20260809 04:44:17.570966 27519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe1cf6cb81f42b391b5efba511f3683" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40545 } }
I20260809 04:44:17.571118 27519 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:17.571189 27519 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:17.571310 27519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:17.572178 27519 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe1cf6cb81f42b391b5efba511f3683" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40545 } }
I20260809 04:44:17.572541 27519 leader_election.cc:302] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fe1cf6cb81f42b391b5efba511f3683; no voters:
I20260809 04:44:17.572922 27519 leader_election.cc:288] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:17.573026 27524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:17.573264 27524 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe1cf6cb81f42b391b5efba511f3683, State: Running, Role: LEADER
I20260809 04:44:17.573581 27524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe1cf6cb81f42b391b5efba511f3683" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40545 } }
I20260809 04:44:17.573982 27522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:17.560259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44926 (local address 127.5.222.254:40545)
0809 04:44:17.560707 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:17.560722 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:17.560758 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:44:17.561453 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:17.561472 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:17.561786 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:17.562120 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:17.562952 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:17.563966 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:17.564944 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:17.565470 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:17.565645 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:17.568233 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:17.568237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:17.568249 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:17.568274 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:17.570739 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:17.571243 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:17.571248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:17.571253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:17.571434 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:17.571716 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:17.571720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:17.571721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:17.572121 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:17.572339 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:17.572640 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 04:44:17.572964 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":140,"threads_started":1}
I20260809 04:44:17.574268 27519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:17.574786 27524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fe1cf6cb81f42b391b5efba511f3683. Latest consensus state: current_term: 1 leader_uuid: "8fe1cf6cb81f42b391b5efba511f3683" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe1cf6cb81f42b391b5efba511f3683" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40545 } } }
I20260809 04:44:17.574896 27524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:17.575240 27525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fe1cf6cb81f42b391b5efba511f3683" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe1cf6cb81f42b391b5efba511f3683" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 40545 } } }
I20260809 04:44:17.575351 27525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:17.576421 27528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:17.579548 27528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:17.585323 27528 catalog_manager.cc:1347] Generated new cluster ID: 816641fb7f1245faac8d5b5b3ab7788d
I20260809 04:44:17.585398 27528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:17.630978 27528 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:17.631990 27528 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:17.638532 27528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683: Generated new TSK 0
I20260809 04:44:17.639384 27528 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.222.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
Entry for principal kudu/127.5.222.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab.
I20260809 04:44:17.680092 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40545
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:17.796326 27547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:17.796605 27547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:17.796649 27547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:17.800868 27547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:17.800931 27547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:17.801044 27547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:44:17.806689 27547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.193.keytab
--principal=kudu/127.5.222.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:17.807744 27547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:17.808772 27547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:17.818625 27553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:17.818593 27556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:17.818646 27547 server_base.cc:1034] running on GCE node
W20260809 04:44:17.818593 27554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:17.819334 27547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:17.820004 27547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:17.821238 27547 hybrid_clock.cc:648] HybridClock initialized: now 1786250657821207 us; error 41 us; skew 500 ppm
Aug 09 04:44:17 dist-test-slave-vxj2 krb5kdc[27434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:17.824041 27547 init.cc:377] Logged in from keytab as kudu/127.5.222.193@KRBTEST.COM (short username kudu)
I20260809 04:44:17.824939 27547 webserver.cc:460] Webserver started at http://127.5.222.193:34377/ using document root <none> and password file <none>
I20260809 04:44:17.825565 27547 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:17.825629 27547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:17.825868 27547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:17.828027 27547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "d5fdb3046d754cb693444aaa1356611c"
format_stamp: "Formatted at 2026-08-09 04:44:17 on dist-test-slave-vxj2"
server_key: "27a964d94314ce02453e95ade5383e0d"
server_key_iv: "947789931dfac93928bb4293602a03be"
server_key_version: "encryptionkey@0"
I20260809 04:44:17.828590 27547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "d5fdb3046d754cb693444aaa1356611c"
format_stamp: "Formatted at 2026-08-09 04:44:17 on dist-test-slave-vxj2"
server_key: "27a964d94314ce02453e95ade5383e0d"
server_key_iv: "947789931dfac93928bb4293602a03be"
server_key_version: "encryptionkey@0"
I20260809 04:44:17.832427 27547 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 04:44:17.834890 27563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:17.836077 27547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:44:17.836208 27547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "d5fdb3046d754cb693444aaa1356611c"
format_stamp: "Formatted at 2026-08-09 04:44:17 on dist-test-slave-vxj2"
server_key: "27a964d94314ce02453e95ade5383e0d"
server_key_iv: "947789931dfac93928bb4293602a03be"
server_key_version: "encryptionkey@0"
I20260809 04:44:17.836323 27547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:17.865876 27547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:17.869313 27547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:17.869529 27547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:17.870153 27547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:17.871007 27547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:17.871079 27547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:17.871153 27547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:17.871196 27547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:17.880823 27547 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:44441
I20260809 04:44:17.880856 27683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:44441 every 8 connection(s)
I20260809 04:44:17.881727 27547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:17.887669 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 27547
I20260809 04:44:17.887813 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:17.888132 6011 external_mini_cluster.cc:1428] Setting key 0d834ef3693ee4286f14bf87cf121427
Aug 09 04:44:17 dist-test-slave-vxj2 krb5kdc[27434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.193@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:17.896018 27522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:17.883575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:43345 (local address 127.5.222.254:40545)
0809 04:44:17.883728 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:17.883732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:17.884622 (+ 890us) server_negotiation.cc:407] Connection header received
0809 04:44:17.885505 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:17.885508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:17.885554 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:17.885628 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:17.886789 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:17.887305 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:17.888842 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:17.889103 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:17.889241 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:17.891964 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:17.891970 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:17.891974 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:17.892007 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:17.893542 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:17.894172 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:17.894174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:17.894175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:17.894250 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:17.894680 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:17.894682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:17.894684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:17.894858 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:17.894946 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:17.895604 (+ 658us) server_negotiation.cc:299] Negotiation successful
0809 04:44:17.895757 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:44:17.896682 27686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:17.884008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40545 (local address 127.5.222.193:43345)
0809 04:44:17.884463 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:17.884506 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:17.885315 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:17.885793 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:17.885801 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:17.886171 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:17.886651 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:17.886665 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:17.887434 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:17.887437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:17.888730 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:17.888740 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:17.889295 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:17.889299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:17.889493 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:17.890078 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:17.890102 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:17.891791 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:17.893659 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:17.893668 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:17.893686 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:17.894061 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:17.894346 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:17.894353 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:17.894357 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:17.894558 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:17.894973 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:17.894980 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:17.895312 (+ 332us) client_negotiation.cc:769] Sending connection context
0809 04:44:17.895512 (+ 200us) client_negotiation.cc:240] Negotiation successful
0809 04:44:17.895773 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":113,"threads_started":1}
I20260809 04:44:17.898025 27684 heartbeater.cc:347] Connected to a master server at 127.5.222.254:40545
I20260809 04:44:17.898254 27684 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:17.898733 27684 heartbeater.cc:511] Master 127.5.222.254:40545 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.222.194@KRBTEST.COM; defaulting to no policy
I20260809 04:44:17.900324 27481 ts_manager.cc:194] Registered new tserver with Master: d5fdb3046d754cb693444aaa1356611c (127.5.222.193:44441)
I20260809 04:44:17.901562 27481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.193@KRBTEST.COM'} at 127.5.222.193:43345
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
Entry for principal kudu/127.5.222.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab.
I20260809 04:44:17.918008 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40545
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:18.030604 27689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:18.030879 27689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:18.030958 27689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:18.035224 27689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:18.035284 27689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:18.035372 27689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:44:18.040782 27689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.194.keytab
--principal=kudu/127.5.222.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:18.041880 27689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:18.042923 27689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:18.052910 27695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:18.052981 27694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:18.052927 27697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:18.053330 27689 server_base.cc:1034] running on GCE node
I20260809 04:44:18.053781 27689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:18.054487 27689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:18.055725 27689 hybrid_clock.cc:648] HybridClock initialized: now 1786250658055703 us; error 29 us; skew 500 ppm
Aug 09 04:44:18 dist-test-slave-vxj2 krb5kdc[27434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250658, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:18.059255 27689 init.cc:377] Logged in from keytab as kudu/127.5.222.194@KRBTEST.COM (short username kudu)
I20260809 04:44:18.060231 27689 webserver.cc:460] Webserver started at http://127.5.222.194:37411/ using document root <none> and password file <none>
I20260809 04:44:18.060879 27689 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:18.061000 27689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:18.061335 27689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:18.063660 27689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "d2e3e4ec319c42f1b9878e9142194359"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-vxj2"
server_key: "33ef248697a25074b1ac2ba6b8b213a7"
server_key_iv: "732ba3f1785b5ff96cdd33e7810c3289"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.064316 27689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "d2e3e4ec319c42f1b9878e9142194359"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-vxj2"
server_key: "33ef248697a25074b1ac2ba6b8b213a7"
server_key_iv: "732ba3f1785b5ff96cdd33e7810c3289"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.068485 27689 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:44:18.071147 27704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.072535 27689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:44:18.072697 27689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "d2e3e4ec319c42f1b9878e9142194359"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-vxj2"
server_key: "33ef248697a25074b1ac2ba6b8b213a7"
server_key_iv: "732ba3f1785b5ff96cdd33e7810c3289"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.072824 27689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:18.097018 27689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:18.100822 27689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:18.101061 27689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:18.101749 27689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:18.102685 27689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:18.102766 27689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.102850 27689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:18.102907 27689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.112396 27689 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:36651
I20260809 04:44:18.112417 27827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:36651 every 8 connection(s)
I20260809 04:44:18.113464 27689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:18.115387 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 27689
I20260809 04:44:18.115540 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:18.115904 6011 external_mini_cluster.cc:1428] Setting key 19c50eacbd887a5e9b86018c9298398d
Aug 09 04:44:18 dist-test-slave-vxj2 krb5kdc[27434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250658, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.194@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.222.195@KRBTEST.COM; defaulting to no policy
I20260809 04:44:18.128356 27522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.116432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:45687 (local address 127.5.222.254:40545)
0809 04:44:18.116582 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:18.116587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:18.116602 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:44:18.117600 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:18.117602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:18.117648 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:18.117722 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:18.119400 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.120329 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.121558 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.121791 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.121908 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.124462 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:18.124471 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:18.124473 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:18.124496 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:18.126044 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:18.126662 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:18.126669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:18.126671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:18.126757 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:18.127115 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:18.127118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:18.127120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:18.127335 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:18.127475 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:18.128040 (+ 565us) server_negotiation.cc:299] Negotiation successful
0809 04:44:18.128168 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.195@KRBTEST.COM" created.
I20260809 04:44:18.129208 27830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.115736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40545 (local address 127.5.222.194:45687)
0809 04:44:18.116275 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:18.116345 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:18.117348 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:18.117904 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:18.117915 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:18.118467 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:18.119174 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.119197 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.120470 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.120472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.121424 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.121440 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.121949 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.121954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.122178 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.122830 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:18.122847 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:18.124289 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:18.126167 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:18.126172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:18.126188 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:18.126532 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:18.126851 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:18.126853 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:18.126854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:18.127003 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:18.127473 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:18.127480 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:18.127759 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 04:44:18.127998 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 04:44:18.128307 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":144,"threads_started":1}
I20260809 04:44:18.130414 27828 heartbeater.cc:347] Connected to a master server at 127.5.222.254:40545
I20260809 04:44:18.130679 27828 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:18.131168 27828 heartbeater.cc:511] Master 127.5.222.254:40545 requested a full tablet report, sending...
I20260809 04:44:18.132248 27481 ts_manager.cc:194] Registered new tserver with Master: d2e3e4ec319c42f1b9878e9142194359 (127.5.222.194:36651)
I20260809 04:44:18.133177 27481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.194@KRBTEST.COM'} at 127.5.222.194:45687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
Entry for principal kudu/127.5.222.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab.
I20260809 04:44:18.144204 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40545
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:33257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:18.256160 27834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:18.256410 27834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:18.256453 27834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:18.260921 27834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:18.260984 27834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:18.261085 27834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:44:18.266335 27834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:33257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.195.keytab
--principal=kudu/127.5.222.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:40545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:18.267390 27834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:18.268397 27834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:18.278995 27840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:18.279008 27841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:18.279021 27843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:18.279366 27834 server_base.cc:1034] running on GCE node
I20260809 04:44:18.279843 27834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:18.280467 27834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:18.281651 27834 hybrid_clock.cc:648] HybridClock initialized: now 1786250658281634 us; error 32 us; skew 500 ppm
Aug 09 04:44:18 dist-test-slave-vxj2 krb5kdc[27434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250658, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:18.284392 27834 init.cc:377] Logged in from keytab as kudu/127.5.222.195@KRBTEST.COM (short username kudu)
I20260809 04:44:18.285274 27834 webserver.cc:460] Webserver started at http://127.5.222.195:45129/ using document root <none> and password file <none>
I20260809 04:44:18.285849 27834 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:18.285938 27834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:18.286168 27834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:18.288169 27834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "abc2a896289e4682b4ebdfd52c1d4446"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-vxj2"
server_key: "ecf3d8324aba6c0c4790dc8e9df7aad7"
server_key_iv: "9ca1601394c5559ca3b850c68f886d3a"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.288738 27834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "abc2a896289e4682b4ebdfd52c1d4446"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-vxj2"
server_key: "ecf3d8324aba6c0c4790dc8e9df7aad7"
server_key_iv: "9ca1601394c5559ca3b850c68f886d3a"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.292317 27834 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 04:44:18.294687 27851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.295889 27834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:44:18.296023 27834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "abc2a896289e4682b4ebdfd52c1d4446"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-vxj2"
server_key: "ecf3d8324aba6c0c4790dc8e9df7aad7"
server_key_iv: "9ca1601394c5559ca3b850c68f886d3a"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.296132 27834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:18.328651 27834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:18.332020 27834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:18.332226 27834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:18.332821 27834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:18.333746 27834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:18.333793 27834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.333861 27834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:18.333900 27834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.342684 27834 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:35841
I20260809 04:44:18.342784 27968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:35841 every 8 connection(s)
I20260809 04:44:18.343587 27834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:18.351029 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 27834
I20260809 04:44:18.351176 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:18.351467 6011 external_mini_cluster.cc:1428] Setting key c6d9f218609046266dbaf6a4b7dd80fd
Aug 09 04:44:18 dist-test-slave-vxj2 krb5kdc[27434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250658, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.195@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:18.356601 27522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.345408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:46343 (local address 127.5.222.254:40545)
0809 04:44:18.345574 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:18.345579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:18.346436 (+ 857us) server_negotiation.cc:407] Connection header received
0809 04:44:18.347280 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:18.347284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:18.347330 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:18.347411 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:18.348511 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.349038 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.350169 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.350402 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.350493 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.353443 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:18.353447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:18.353449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:18.353476 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:18.354848 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:18.355325 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:18.355327 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:18.355329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:18.355405 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:18.355696 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:18.355698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:18.355699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:18.355876 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:18.355953 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:18.356392 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 04:44:18.356485 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:44:18.357414 27972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.345834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:40545 (local address 127.5.222.195:46343)
0809 04:44:18.346276 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:18.346319 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:18.347066 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:18.347566 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:18.347573 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:18.347904 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:18.348364 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.348377 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.349148 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.349191 (+ 43us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.350049 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.350058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.350533 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.350536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.350666 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.351377 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:18.351400 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:18.353291 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:18.354954 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:18.354959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:18.354969 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:18.355216 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:18.355482 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:18.355484 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:18.355485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:18.355610 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:18.355965 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:18.355970 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:18.356173 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 04:44:18.356328 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 04:44:18.356563 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":120,"threads_started":1}
I20260809 04:44:18.358440 27969 heartbeater.cc:347] Connected to a master server at 127.5.222.254:40545
I20260809 04:44:18.358630 27969 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:18.359045 27969 heartbeater.cc:511] Master 127.5.222.254:40545 requested a full tablet report, sending...
I20260809 04:44:18.359926 27481 ts_manager.cc:194] Registered new tserver with Master: abc2a896289e4682b4ebdfd52c1d4446 (127.5.222.195:35841)
I20260809 04:44:18.360486 27481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.222.195@KRBTEST.COM'} at 127.5.222.195:46343
I20260809 04:44:18.365483 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:44:18.374624 27522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.367534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44934 (local address 127.5.222.254:40545)
0809 04:44:18.367648 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:18.367651 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:18.367844 (+ 193us) server_negotiation.cc:407] Connection header received
0809 04:44:18.367939 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:18.367942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:18.367976 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:18.368043 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:18.368741 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.369279 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.370123 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.370322 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.370395 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.371703 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:18.371706 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:18.371708 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:18.371730 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:18.373003 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:18.373469 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:18.373471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:18.373473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:18.373547 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:18.373819 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:18.373821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:18.373822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:18.374017 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:18.374217 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:18.374404 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 04:44:18.374504 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:44:18.377609 27481 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:18.380831 27481 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:18.395231 27985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.389214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:44441 (local address 127.0.0.1:51220)
0809 04:44:18.389735 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:18.389794 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:18.389923 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:18.390558 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:18.390564 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:18.390588 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:18.390729 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.390737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.392168 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.392171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.393392 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.393402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.394591 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.394594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.394776 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.394820 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 04:44:18.394948 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 04:44:18.395074 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":22,"thread_start_us":96,"threads_started":1}
I20260809 04:44:18.395247 27984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.389393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.5.222.193:44441)
0809 04:44:18.389781 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:18.389801 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:18.390087 (+ 286us) server_negotiation.cc:407] Connection header received
0809 04:44:18.390172 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:18.390178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:18.390383 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:18.390553 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:18.390872 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.392005 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.393660 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.394459 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.394581 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.394743 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:18.394985 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 04:44:18.395079 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":82,"threads_started":1}
I20260809 04:44:18.395692 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.389390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:35841 (local address 127.0.0.1:35764)
0809 04:44:18.389929 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:18.389942 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:18.390064 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:18.391452 (+ 1388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:18.391455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:18.391472 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:18.391562 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.391566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.392790 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.392794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.394018 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.394026 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.395296 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.395299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.395400 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.395422 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:44:18.395497 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 04:44:18.395564 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":104,"threads_started":1}
I20260809 04:44:18.395787 27988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.390382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35764 (local address 127.5.222.195:35841)
0809 04:44:18.391009 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:18.391021 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:18.391042 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:44:18.391114 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:18.391118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:18.391320 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:18.391469 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:18.391685 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.392642 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.394147 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.394993 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.395091 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.395223 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:18.395530 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 04:44:18.395621 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":157,"threads_started":1}
I20260809 04:44:18.396680 27986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.389931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:36651 (local address 127.0.0.1:56510)
0809 04:44:18.390173 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:18.390185 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:18.390302 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:18.392321 (+ 2019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:18.392323 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:18.392338 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:18.392497 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.392502 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.393980 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.393985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.395162 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.395173 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.396294 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.396298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.396384 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.396404 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:44:18.396487 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 04:44:18.396555 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":107,"thread_start_us":90,"threads_started":1}
I20260809 04:44:18.396733 27987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.390371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.5.222.194:36651)
0809 04:44:18.391760 (+ 1389us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:18.391770 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:18.391791 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:44:18.391861 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:18.391866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:18.392023 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:18.392160 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:18.392621 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.393829 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.395298 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.396164 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.396277 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.396409 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:18.396514 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 04:44:18.396599 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1289,"thread_start_us":149,"threads_started":1}
I20260809 04:44:18.398806 27614 tablet_service.cc:1467] Processing CreateTablet for tablet 737dd9f5b4044e69b160753f67adf6fa (DEFAULT_TABLE table=test-table [id=2f2acf205aeb45018748cfc06ab6c989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:18.398808 27900 tablet_service.cc:1467] Processing CreateTablet for tablet 737dd9f5b4044e69b160753f67adf6fa (DEFAULT_TABLE table=test-table [id=2f2acf205aeb45018748cfc06ab6c989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:18.399183 27760 tablet_service.cc:1467] Processing CreateTablet for tablet 737dd9f5b4044e69b160753f67adf6fa (DEFAULT_TABLE table=test-table [id=2f2acf205aeb45018748cfc06ab6c989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:18.399959 27614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 737dd9f5b4044e69b160753f67adf6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:18.399976 27900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 737dd9f5b4044e69b160753f67adf6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:18.400241 27760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 737dd9f5b4044e69b160753f67adf6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:18.406482 27989 tablet_bootstrap.cc:492] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359: Bootstrap starting.
I20260809 04:44:18.407593 27990 tablet_bootstrap.cc:492] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c: Bootstrap starting.
I20260809 04:44:18.408605 27989 tablet_bootstrap.cc:654] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:18.409474 27990 tablet_bootstrap.cc:654] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:18.409835 27989 log.cc:824] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:18.410334 27990 log.cc:824] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:18.410563 27991 tablet_bootstrap.cc:492] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446: Bootstrap starting.
I20260809 04:44:18.412025 27989 tablet_bootstrap.cc:492] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359: No bootstrap required, opened a new log
I20260809 04:44:18.412217 27990 tablet_bootstrap.cc:492] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c: No bootstrap required, opened a new log
I20260809 04:44:18.412354 27989 ts_tablet_manager.cc:1397] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:44:18.412354 27991 tablet_bootstrap.cc:654] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:18.412540 27990 ts_tablet_manager.cc:1397] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:44:18.413487 27991 log.cc:824] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:18.415328 27991 tablet_bootstrap.cc:492] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446: No bootstrap required, opened a new log
I20260809 04:44:18.415524 27991 ts_tablet_manager.cc:1397] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:44:18.416747 27989 raft_consensus.cc:348] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5fdb3046d754cb693444aaa1356611c" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 44441 } } peers { permanent_uuid: "abc2a896289e4682b4ebdfd52c1d4446" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 35841 } } peers { permanent_uuid: "d2e3e4ec319c42f1b9878e9142194359" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36651 } }
I20260809 04:44:18.416842 27990 raft_consensus.cc:348] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5fdb3046d754cb693444aaa1356611c" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 44441 } } peers { permanent_uuid: "abc2a896289e4682b4ebdfd52c1d4446" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 35841 } } peers { permanent_uuid: "d2e3e4ec319c42f1b9878e9142194359" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36651 } }
I20260809 04:44:18.417049 27990 raft_consensus.cc:374] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:18.417050 27989 raft_consensus.cc:374] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:18.417109 27989 raft_consensus.cc:729] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e3e4ec319c42f1b9878e9142194359, State: Initialized, Role: FOLLOWER
I20260809 04:44:18.417140 27990 raft_consensus.cc:729] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fdb3046d754cb693444aaa1356611c, State: Initialized, Role: FOLLOWER
I20260809 04:44:18.417681 27989 consensus_queue.cc:260] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5fdb3046d754cb693444aaa1356611c" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 44441 } } peers { permanent_uuid: "abc2a896289e4682b4ebdfd52c1d4446" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 35841 } } peers { permanent_uuid: "d2e3e4ec319c42f1b9878e9142194359" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36651 } }
I20260809 04:44:18.417666 27990 consensus_queue.cc:260] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5fdb3046d754cb693444aaa1356611c" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 44441 } } peers { permanent_uuid: "abc2a896289e4682b4ebdfd52c1d4446" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 35841 } } peers { permanent_uuid: "d2e3e4ec319c42f1b9878e9142194359" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36651 } }
I20260809 04:44:18.418404 27684 heartbeater.cc:503] Master 127.5.222.254:40545 was elected leader, sending a full tablet report...
I20260809 04:44:18.418726 27989 ts_tablet_manager.cc:1428] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 04:44:18.418884 27991 raft_consensus.cc:348] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5fdb3046d754cb693444aaa1356611c" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 44441 } } peers { permanent_uuid: "abc2a896289e4682b4ebdfd52c1d4446" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 35841 } } peers { permanent_uuid: "d2e3e4ec319c42f1b9878e9142194359" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36651 } }
I20260809 04:44:18.419076 27991 raft_consensus.cc:374] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:18.419109 27991 raft_consensus.cc:729] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abc2a896289e4682b4ebdfd52c1d4446, State: Initialized, Role: FOLLOWER
I20260809 04:44:18.419509 27828 heartbeater.cc:503] Master 127.5.222.254:40545 was elected leader, sending a full tablet report...
I20260809 04:44:18.419486 27991 consensus_queue.cc:260] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5fdb3046d754cb693444aaa1356611c" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 44441 } } peers { permanent_uuid: "abc2a896289e4682b4ebdfd52c1d4446" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 35841 } } peers { permanent_uuid: "d2e3e4ec319c42f1b9878e9142194359" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36651 } }
I20260809 04:44:18.420095 27969 heartbeater.cc:503] Master 127.5.222.254:40545 was elected leader, sending a full tablet report...
I20260809 04:44:18.420442 27991 ts_tablet_manager.cc:1428] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:44:18.422115 27990 ts_tablet_manager.cc:1428] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c: Time spent starting tablet: real 0.010s user 0.004s sys 0.004s
I20260809 04:44:18.493022 27998 raft_consensus.cc:482] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:18.493336 27998 raft_consensus.cc:504] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5fdb3046d754cb693444aaa1356611c" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 44441 } } peers { permanent_uuid: "abc2a896289e4682b4ebdfd52c1d4446" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 35841 } } peers { permanent_uuid: "d2e3e4ec319c42f1b9878e9142194359" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36651 } }
I20260809 04:44:18.494473 27998 leader_election.cc:288] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5fdb3046d754cb693444aaa1356611c (127.5.222.193:44441), d2e3e4ec319c42f1b9878e9142194359 (127.5.222.194:36651)
I20260809 04:44:18.498799 27972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.494816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:44441 (local address 127.5.222.195:47779)
0809 04:44:18.495002 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:18.495021 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:18.495135 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:18.495548 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:18.495551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:18.495569 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:18.495720 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.495729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.496640 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.496644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.497552 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.497562 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.498347 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.498350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.498457 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.498483 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:44:18.498568 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 04:44:18.498653 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 04:44:18.498874 27984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.494923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:47779 (local address 127.5.222.193:44441)
0809 04:44:18.495072 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:18.495078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:18.495098 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:44:18.495284 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:18.495290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:18.495351 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:18.495421 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:18.495853 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.496519 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.497679 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.498230 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.498327 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.498366 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:18.498588 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 04:44:18.498684 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:44:18.499145 27987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.495334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:52899 (local address 127.5.222.194:36651)
0809 04:44:18.495477 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:18.495481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:18.495657 (+ 176us) server_negotiation.cc:407] Connection header received
0809 04:44:18.495758 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:18.495764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:18.495827 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:18.495905 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:18.496214 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.496815 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.498034 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.498554 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.498661 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.498706 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:18.498934 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 04:44:18.499014 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:44:18.499487 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.495124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:36651 (local address 127.5.222.195:52899)
0809 04:44:18.495545 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:18.495559 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:18.495668 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:18.495947 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:18.495950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:18.495967 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:18.496096 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.496103 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.496938 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.496941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.497906 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.497917 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.498676 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.498679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.498773 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.498797 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:44:18.499304 (+ 507us) client_negotiation.cc:240] Negotiation successful
0809 04:44:18.499389 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":133,"threads_started":1}
I20260809 04:44:18.499642 27636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737dd9f5b4044e69b160753f67adf6fa" candidate_uuid: "abc2a896289e4682b4ebdfd52c1d4446" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fdb3046d754cb693444aaa1356611c" is_pre_election: true
I20260809 04:44:18.499984 27636 raft_consensus.cc:2454] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abc2a896289e4682b4ebdfd52c1d4446 in term 0.
I20260809 04:44:18.500141 27781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737dd9f5b4044e69b160753f67adf6fa" candidate_uuid: "abc2a896289e4682b4ebdfd52c1d4446" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e3e4ec319c42f1b9878e9142194359" is_pre_election: true
I20260809 04:44:18.500461 27781 raft_consensus.cc:2454] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abc2a896289e4682b4ebdfd52c1d4446 in term 0.
I20260809 04:44:18.500501 27852 leader_election.cc:302] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abc2a896289e4682b4ebdfd52c1d4446, d5fdb3046d754cb693444aaa1356611c; no voters:
I20260809 04:44:18.500785 27998 raft_consensus.cc:2781] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:18.500875 27998 raft_consensus.cc:482] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:18.500919 27998 raft_consensus.cc:3037] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:18.502005 27998 raft_consensus.cc:504] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5fdb3046d754cb693444aaa1356611c" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 44441 } } peers { permanent_uuid: "abc2a896289e4682b4ebdfd52c1d4446" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 35841 } } peers { permanent_uuid: "d2e3e4ec319c42f1b9878e9142194359" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36651 } }
I20260809 04:44:18.502341 27998 leader_election.cc:288] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [CANDIDATE]: Term 1 election: Requested vote from peers d5fdb3046d754cb693444aaa1356611c (127.5.222.193:44441), d2e3e4ec319c42f1b9878e9142194359 (127.5.222.194:36651)
I20260809 04:44:18.502727 27781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737dd9f5b4044e69b160753f67adf6fa" candidate_uuid: "abc2a896289e4682b4ebdfd52c1d4446" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e3e4ec319c42f1b9878e9142194359"
I20260809 04:44:18.502727 27636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737dd9f5b4044e69b160753f67adf6fa" candidate_uuid: "abc2a896289e4682b4ebdfd52c1d4446" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fdb3046d754cb693444aaa1356611c"
I20260809 04:44:18.502853 27636 raft_consensus.cc:3037] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:18.502853 27781 raft_consensus.cc:3037] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:18.504156 27636 raft_consensus.cc:2454] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abc2a896289e4682b4ebdfd52c1d4446 in term 1.
I20260809 04:44:18.504156 27781 raft_consensus.cc:2454] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abc2a896289e4682b4ebdfd52c1d4446 in term 1.
I20260809 04:44:18.504456 27852 leader_election.cc:302] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abc2a896289e4682b4ebdfd52c1d4446, d2e3e4ec319c42f1b9878e9142194359; no voters:
I20260809 04:44:18.504637 27998 raft_consensus.cc:2781] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:18.504879 27998 raft_consensus.cc:686] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 1 LEADER]: Becoming Leader. State: Replica: abc2a896289e4682b4ebdfd52c1d4446, State: Running, Role: LEADER
I20260809 04:44:18.505270 27998 consensus_queue.cc:237] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5fdb3046d754cb693444aaa1356611c" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 44441 } } peers { permanent_uuid: "abc2a896289e4682b4ebdfd52c1d4446" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 35841 } } peers { permanent_uuid: "d2e3e4ec319c42f1b9878e9142194359" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36651 } }
I20260809 04:44:18.508512 27481 catalog_manager.cc:5495] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 reported cstate change: term changed from 0 to 1, leader changed from <none> to abc2a896289e4682b4ebdfd52c1d4446 (127.5.222.195). New cstate: current_term: 1 leader_uuid: "abc2a896289e4682b4ebdfd52c1d4446" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5fdb3046d754cb693444aaa1356611c" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 44441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abc2a896289e4682b4ebdfd52c1d4446" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 35841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e3e4ec319c42f1b9878e9142194359" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36651 } health_report { overall_health: UNKNOWN } } }
I20260809 04:44:18.528761 27988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.524587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35770 (local address 127.5.222.195:35841)
0809 04:44:18.524748 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:18.524753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:18.524769 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:44:18.524976 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:18.524978 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:18.525035 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:18.525147 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:18.525440 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.525953 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.527047 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.527437 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.527558 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.527932 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:18.528363 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:18.528500 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:18.528563 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 04:44:18.528621 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:44:18.535233 27636 raft_consensus.cc:1261] T 737dd9f5b4044e69b160753f67adf6fa P d5fdb3046d754cb693444aaa1356611c [term 1 FOLLOWER]: Refusing update from remote peer abc2a896289e4682b4ebdfd52c1d4446: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:18.535233 27781 raft_consensus.cc:1261] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359 [term 1 FOLLOWER]: Refusing update from remote peer abc2a896289e4682b4ebdfd52c1d4446: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:18.536158 27998 consensus_queue.cc:1035] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e3e4ec319c42f1b9878e9142194359" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 36651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:18.536449 28000 consensus_queue.cc:1035] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fdb3046d754cb693444aaa1356611c" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 44441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:18.543898 28005 mvcc.cc:205] Tried to move back new op lower bound from 7316482697351344128 to 7316482697240813568. Current Snapshot: MvccSnapshot[applied={T|T < 7316482697351344128}]
I20260809 04:44:18.548288 28004 mvcc.cc:205] Tried to move back new op lower bound from 7316482697351344128 to 7316482697240813568. Current Snapshot: MvccSnapshot[applied={T|T < 7316482697351344128 or (T in {7316482697351344128})}]
I20260809 04:44:18.552580 27479 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:18.552794 27479 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:18.555773 27479 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8fe1cf6cb81f42b391b5efba511f3683: Sending DeleteTablet for 3 replicas of tablet 737dd9f5b4044e69b160753f67adf6fa
I20260809 04:44:18.556668 27760 tablet_service.cc:1514] Processing DeleteTablet for tablet 737dd9f5b4044e69b160753f67adf6fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:18 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:56510
I20260809 04:44:18.556769 27900 tablet_service.cc:1514] Processing DeleteTablet for tablet 737dd9f5b4044e69b160753f67adf6fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:18 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:35764
I20260809 04:44:18.557227 27614 tablet_service.cc:1514] Processing DeleteTablet for tablet 737dd9f5b4044e69b160753f67adf6fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:18 UTC) from {username='kudu', principal='kudu/127.5.222.254@KRBTEST.COM'} at 127.0.0.1:51220
I20260809 04:44:18.557448 28012 tablet_replica.cc:330] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359: stopping tablet replica
I20260809 04:44:18.558368 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 27547
I20260809 04:44:18.564311 28012 raft_consensus.cc:2229] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:18.564883 28012 raft_consensus.cc:2258] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:18.566111 28013 tablet_replica.cc:330] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446: stopping tablet replica
I20260809 04:44:18.566540 28013 raft_consensus.cc:2229] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:18.566809 28012 ts_tablet_manager.cc:1905] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:18.567039 28013 raft_consensus.cc:2258] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:18.570020 28012 ts_tablet_manager.cc:1918] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:18.571663 28013 ts_tablet_manager.cc:1905] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:18.572844 28012 log.cc:1196] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/737dd9f5b4044e69b160753f67adf6fa
I20260809 04:44:18.573299 28012 ts_tablet_manager.cc:1939] T 737dd9f5b4044e69b160753f67adf6fa P d2e3e4ec319c42f1b9878e9142194359: Deleting consensus metadata
W20260809 04:44:18.573460 27464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:44441 (error 108)
I20260809 04:44:18.573839 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 27689
I20260809 04:44:18.575279 28013 ts_tablet_manager.cc:1918] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:18.576016 27985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.575590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:44441 (local address 127.0.0.1:51224)
0809 04:44:18.575819 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:18.575897 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:44441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260809 04:44:18.576220 27464 catalog_manager.cc:4584] TS d5fdb3046d754cb693444aaa1356611c (127.5.222.193:44441): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 737dd9f5b4044e69b160753f67adf6fa: Network error: Client connection negotiation failed: client connection to 127.5.222.193:44441: connect: Connection refused (error 111)
I20260809 04:44:18.575371 28013 log.cc:1196] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/737dd9f5b4044e69b160753f67adf6fa
I20260809 04:44:18.576782 28013 ts_tablet_manager.cc:1939] T 737dd9f5b4044e69b160753f67adf6fa P abc2a896289e4682b4ebdfd52c1d4446: Deleting consensus metadata
I20260809 04:44:18.578261 27465 catalog_manager.cc:4841] TS abc2a896289e4682b4ebdfd52c1d4446 (127.5.222.195:35841): tablet 737dd9f5b4044e69b160753f67adf6fa (table test-table [id=2f2acf205aeb45018748cfc06ab6c989]) successfully deleted
I20260809 04:44:18.582650 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 27834
I20260809 04:44:18.583169 27985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.582819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:36651 (local address 127.0.0.1:56522)
0809 04:44:18.583018 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:18.583077 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.194:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 04:44:18.590314 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 27448
2026-08-09T04:44:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2450 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T04:44:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:18Z Disabled control of system clock
I20260809 04:44:18.626111 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:46151
--webserver_interface=127.5.222.254
--webserver_port=0
--builtin_ntp_servers=127.5.222.212:46439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:46151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:44:18.741086 28019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:18.741384 28019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:18.741420 28019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:18.745354 28019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:18.745412 28019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:18.745427 28019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:18.745445 28019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:18.745460 28019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:18.750136 28019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:46151
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:46151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:18.751128 28019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:18.752521 28019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:18.760842 28028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:18.760892 28025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:18.761068 28026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:18.761094 28019 server_base.cc:1034] running on GCE node
I20260809 04:44:18.761824 28019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:18.763075 28019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:18.764300 28019 hybrid_clock.cc:648] HybridClock initialized: now 1786250658764283 us; error 36 us; skew 500 ppm
I20260809 04:44:18.766196 28019 webserver.cc:460] Webserver started at http://127.5.222.254:38223/ using document root <none> and password file <none>
I20260809 04:44:18.766822 28019 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:18.766925 28019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:18.767172 28019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:18.769407 28019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "992983cf7a944d5d97c8f55dc23ed17e"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-vxj2"
server_key: "95bf8f2f3585e9df80cdef24fc51ac50"
server_key_iv: "e8c9b9e9112e936314c0f514d0078c71"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.770023 28019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "992983cf7a944d5d97c8f55dc23ed17e"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-vxj2"
server_key: "95bf8f2f3585e9df80cdef24fc51ac50"
server_key_iv: "e8c9b9e9112e936314c0f514d0078c71"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.773880 28019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:44:18.776335 28034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.777608 28019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:18.777755 28019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "992983cf7a944d5d97c8f55dc23ed17e"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-vxj2"
server_key: "95bf8f2f3585e9df80cdef24fc51ac50"
server_key_iv: "e8c9b9e9112e936314c0f514d0078c71"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.777868 28019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:18.791401 28019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:18.792094 28019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:18.792317 28019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:18.799687 28019 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:46151
I20260809 04:44:18.799702 28087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:46151 every 8 connection(s)
I20260809 04:44:18.800659 28019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:18.802292 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 28019
I20260809 04:44:18.802434 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:18.802819 6011 external_mini_cluster.cc:1428] Setting key bf95a5051fafc3f5aae7c50ed67b867a
I20260809 04:44:18.804370 28088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:18.810425 28088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e: Bootstrap starting.
I20260809 04:44:18.812773 28088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:18.813537 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.804466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57804 (local address 127.5.222.254:46151)
0809 04:44:18.805226 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:18.805250 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:18.805298 (+ 48us) server_negotiation.cc:407] Connection header received
0809 04:44:18.806574 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:18.806595 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:18.806998 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:18.807346 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:18.807795 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.808858 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.809912 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.810420 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.810614 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.811553 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:18.811558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:18.811570 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:18.812045 (+ 475us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:18.812104 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:18.812127 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:18.812248 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:18.812430 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 04:44:18.812691 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":146,"threads_started":1}
I20260809 04:44:18.813577 28088 log.cc:824] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:18.815459 28088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e: No bootstrap required, opened a new log
I20260809 04:44:18.819046 28088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992983cf7a944d5d97c8f55dc23ed17e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 46151 } }
I20260809 04:44:18.819329 28088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:18.819382 28088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992983cf7a944d5d97c8f55dc23ed17e, State: Initialized, Role: FOLLOWER
I20260809 04:44:18.819809 28088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992983cf7a944d5d97c8f55dc23ed17e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 46151 } }
I20260809 04:44:18.819969 28088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:18.820043 28088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:18.820115 28088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:18.820916 28088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992983cf7a944d5d97c8f55dc23ed17e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 46151 } }
I20260809 04:44:18.821300 28088 leader_election.cc:302] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 992983cf7a944d5d97c8f55dc23ed17e; no voters:
I20260809 04:44:18.821552 28088 leader_election.cc:288] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:18.821699 28094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:18.821950 28094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [term 1 LEADER]: Becoming Leader. State: Replica: 992983cf7a944d5d97c8f55dc23ed17e, State: Running, Role: LEADER
I20260809 04:44:18.822268 28094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992983cf7a944d5d97c8f55dc23ed17e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 46151 } }
I20260809 04:44:18.822889 28088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:18.824069 28096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "992983cf7a944d5d97c8f55dc23ed17e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992983cf7a944d5d97c8f55dc23ed17e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 46151 } } }
I20260809 04:44:18.824077 28097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 992983cf7a944d5d97c8f55dc23ed17e. Latest consensus state: current_term: 1 leader_uuid: "992983cf7a944d5d97c8f55dc23ed17e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992983cf7a944d5d97c8f55dc23ed17e" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 46151 } } }
I20260809 04:44:18.824314 28096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:18.824344 28097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:18.825227 28104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:18.828205 28104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:18.833355 28104 catalog_manager.cc:1347] Generated new cluster ID: 50013a008e1646b8b16669a2fdcea3f5
I20260809 04:44:18.833422 28104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:18.844900 28104 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:18.845956 28104 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:18.851573 28104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e: Generated new TSK 0
I20260809 04:44:18.852207 28104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:18.857734 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.193:0
--local_ip_for_outbound_sockets=127.5.222.193
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:46151
--builtin_ntp_servers=127.5.222.212:46439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:44:18.969655 28116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:18.969910 28116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:18.969951 28116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:18.974124 28116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:18.974184 28116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:18.974238 28116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.193
I20260809 04:44:18.979053 28116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.222.193
--webserver_port=0
--tserver_master_addrs=127.5.222.254:46151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.193
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:18.980173 28116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:18.981702 28116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:18.991393 28121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:18.991384 28124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:18.991384 28122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:18.991657 28116 server_base.cc:1034] running on GCE node
I20260809 04:44:18.992067 28116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:18.992687 28116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:18.993854 28116 hybrid_clock.cc:648] HybridClock initialized: now 1786250658993838 us; error 26 us; skew 500 ppm
I20260809 04:44:18.995604 28116 webserver.cc:460] Webserver started at http://127.5.222.193:42309/ using document root <none> and password file <none>
I20260809 04:44:18.996193 28116 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:18.996255 28116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:18.996519 28116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:18.998601 28116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/data/instance:
uuid: "40dc05aa2cc741b5bc52da187f76abd4"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-vxj2"
server_key: "317e11903445b92171869ef8f082a0f9"
server_key_iv: "f9c4aab587251c00a1c55e6064b10639"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.999173 28116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance:
uuid: "40dc05aa2cc741b5bc52da187f76abd4"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-vxj2"
server_key: "317e11903445b92171869ef8f082a0f9"
server_key_iv: "f9c4aab587251c00a1c55e6064b10639"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.003383 28116 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:19.005926 28131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.007439 28116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:19.007578 28116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/wal
uuid: "40dc05aa2cc741b5bc52da187f76abd4"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-vxj2"
server_key: "317e11903445b92171869ef8f082a0f9"
server_key_iv: "f9c4aab587251c00a1c55e6064b10639"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.007704 28116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:19.028568 28116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:19.029393 28116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:19.029620 28116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:19.030238 28116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:19.031213 28116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:19.031262 28116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.031332 28116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:19.031364 28116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.040297 28116 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.193:42059
I20260809 04:44:19.040374 28251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.193:42059 every 8 connection(s)
I20260809 04:44:19.041296 28116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:19.044509 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 28116
I20260809 04:44:19.044651 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:19.044952 6011 external_mini_cluster.cc:1428] Setting key 1b543bba1e6f930b5bacb4d2daa88ad3
I20260809 04:44:19.047016 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.194:0
--local_ip_for_outbound_sockets=127.5.222.194
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:46151
--builtin_ntp_servers=127.5.222.212:46439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:44:19.052201 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.043140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.193:33025 (local address 127.5.222.254:46151)
0809 04:44:19.043289 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.043293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.044475 (+ 1182us) server_negotiation.cc:407] Connection header received
0809 04:44:19.046156 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.046159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.046218 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.046311 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:19.047483 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.048046 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.049893 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.050137 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.050221 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.051599 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:19.051602 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:19.051604 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:19.051618 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:19.051645 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:19.051654 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:19.051724 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.051980 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.052083 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:44:19.053539 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.043541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:46151 (local address 127.5.222.193:33025)
0809 04:44:19.044315 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.044368 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.045934 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.046700 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.046706 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.046793 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:19.047339 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.047354 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.048177 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.048184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.049735 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.049750 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.050303 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.050323 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.050563 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.051244 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:19.051247 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:19.051305 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:19.051400 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:19.051753 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:19.051766 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:19.051785 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.051948 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.052320 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":151,"threads_started":1}
I20260809 04:44:19.054920 28252 heartbeater.cc:347] Connected to a master server at 127.5.222.254:46151
I20260809 04:44:19.055162 28252 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:19.055686 28252 heartbeater.cc:511] Master 127.5.222.254:46151 requested a full tablet report, sending...
I20260809 04:44:19.057250 28052 ts_manager.cc:194] Registered new tserver with Master: 40dc05aa2cc741b5bc52da187f76abd4 (127.5.222.193:42059)
I20260809 04:44:19.058380 28052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.222.193:33025
W20260809 04:44:19.159122 28255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:19.159399 28255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:19.159468 28255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:19.163477 28255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:19.163560 28255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:19.163655 28255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.194
I20260809 04:44:19.168521 28255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.222.194
--webserver_port=0
--tserver_master_addrs=127.5.222.254:46151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.194
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:19.169723 28255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:19.171149 28255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:19.180958 28264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:19.180966 28262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:19.180945 28261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:19.181303 28255 server_base.cc:1034] running on GCE node
I20260809 04:44:19.181844 28255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:19.182571 28255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:19.183749 28255 hybrid_clock.cc:648] HybridClock initialized: now 1786250659183720 us; error 51 us; skew 500 ppm
I20260809 04:44:19.185606 28255 webserver.cc:460] Webserver started at http://127.5.222.194:33293/ using document root <none> and password file <none>
I20260809 04:44:19.186173 28255 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:19.186234 28255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:19.186404 28255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:19.188431 28255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/data/instance:
uuid: "d03632453ada497198778ee86169276c"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-vxj2"
server_key: "d9b4f23b39c9cee3ff52e2f808843037"
server_key_iv: "37496e734b0ef5ebce6ed18208dca1c1"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.188943 28255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance:
uuid: "d03632453ada497198778ee86169276c"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-vxj2"
server_key: "d9b4f23b39c9cee3ff52e2f808843037"
server_key_iv: "37496e734b0ef5ebce6ed18208dca1c1"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.192539 28255 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:44:19.194918 28270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.196099 28255 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 04:44:19.196221 28255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/wal
uuid: "d03632453ada497198778ee86169276c"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-vxj2"
server_key: "d9b4f23b39c9cee3ff52e2f808843037"
server_key_iv: "37496e734b0ef5ebce6ed18208dca1c1"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.196343 28255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:19.218847 28255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:19.219592 28255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:19.219805 28255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:19.220427 28255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:19.221447 28255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:19.221501 28255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.221570 28255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:19.221611 28255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.231321 28255 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.194:44561
I20260809 04:44:19.231352 28390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.194:44561 every 8 connection(s)
I20260809 04:44:19.232235 28255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:19.234249 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 28255
I20260809 04:44:19.234382 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:19.234687 6011 external_mini_cluster.cc:1428] Setting key f39ed81113e3e4c9d578c8d222ae1a1d
I20260809 04:44:19.237069 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.222.195:0
--local_ip_for_outbound_sockets=127.5.222.195
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:46151
--builtin_ntp_servers=127.5.222.212:46439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:44:19.243539 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.234177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:49889 (local address 127.5.222.254:46151)
0809 04:44:19.234365 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.234371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.235550 (+ 1179us) server_negotiation.cc:407] Connection header received
0809 04:44:19.237286 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.237290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.237369 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.237464 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:19.238788 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.239662 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.241491 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.241704 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.241828 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.243018 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:19.243021 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:19.243023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:19.243035 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:19.243053 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:19.243060 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:19.243134 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.243326 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.243397 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:44:19.244477 28393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.234594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:46151 (local address 127.5.222.194:49889)
0809 04:44:19.235363 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.235416 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.237027 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.237673 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.237686 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.237814 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:19.238545 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.238568 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.239871 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.239874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.241349 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.241364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.241882 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.241887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.242051 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.242742 (+ 691us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:19.242744 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:19.242791 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:19.242851 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:19.243159 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:19.243174 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:19.243178 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.243323 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.243706 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":153,"threads_started":1}
I20260809 04:44:19.245633 28391 heartbeater.cc:347] Connected to a master server at 127.5.222.254:46151
I20260809 04:44:19.245908 28391 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:19.246598 28391 heartbeater.cc:511] Master 127.5.222.254:46151 requested a full tablet report, sending...
I20260809 04:44:19.247732 28052 ts_manager.cc:194] Registered new tserver with Master: d03632453ada497198778ee86169276c (127.5.222.194:44561)
I20260809 04:44:19.248342 28052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.222.194:49889
W20260809 04:44:19.352298 28394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:19.352586 28394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:19.352651 28394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:19.356741 28394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:19.356829 28394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:19.356913 28394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.222.195
I20260809 04:44:19.362236 28394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:46439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.222.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.222.195
--webserver_port=0
--tserver_master_addrs=127.5.222.254:46151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.222.195
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:19.363413 28394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:19.364902 28394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:19.375052 28400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:19.375048 28399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:19.375136 28394 server_base.cc:1034] running on GCE node
W20260809 04:44:19.375351 28402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:19.375738 28394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:19.376396 28394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:19.377563 28394 hybrid_clock.cc:648] HybridClock initialized: now 1786250659377546 us; error 31 us; skew 500 ppm
I20260809 04:44:19.379376 28394 webserver.cc:460] Webserver started at http://127.5.222.195:39031/ using document root <none> and password file <none>
I20260809 04:44:19.379943 28394 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:19.380023 28394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:19.380394 28394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:19.383473 28394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/data/instance:
uuid: "2aa5624e70bc4b0983258bf6290f2f6e"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-vxj2"
server_key: "3cb6d2091884dcbbf6b6adc8f4364911"
server_key_iv: "9fdfe1dad41e5dea41cfc2b0a86f7409"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.384294 28394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance:
uuid: "2aa5624e70bc4b0983258bf6290f2f6e"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-vxj2"
server_key: "3cb6d2091884dcbbf6b6adc8f4364911"
server_key_iv: "9fdfe1dad41e5dea41cfc2b0a86f7409"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.388864 28394 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260809 04:44:19.391356 28410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.392550 28394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:44:19.392699 28394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/wal
uuid: "2aa5624e70bc4b0983258bf6290f2f6e"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-vxj2"
server_key: "3cb6d2091884dcbbf6b6adc8f4364911"
server_key_iv: "9fdfe1dad41e5dea41cfc2b0a86f7409"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.392827 28394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:19.406404 28394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:19.407122 28394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:19.407330 28394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:19.408010 28394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:19.408964 28394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:19.409039 28394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.409111 28394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:19.409181 28394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.418622 28394 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.195:34597
I20260809 04:44:19.418645 28530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.195:34597 every 8 connection(s)
I20260809 04:44:19.419549 28394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:19.424964 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 28394
I20260809 04:44:19.425055 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:19.425354 6011 external_mini_cluster.cc:1428] Setting key 169cf82332aef691dc9c87e2de1c633b
I20260809 04:44:19.429394 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.421470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.195:55521 (local address 127.5.222.254:46151)
0809 04:44:19.421616 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.421620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.422692 (+ 1072us) server_negotiation.cc:407] Connection header received
0809 04:44:19.424224 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.424227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.424277 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.424356 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:19.425450 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.426155 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.427550 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.427809 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.427892 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.428928 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:19.428931 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:19.428942 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:19.428954 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:19.428972 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:19.428980 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:19.429038 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.429225 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.429284 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:44:19.430299 28533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.421910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:46151 (local address 127.5.222.195:55521)
0809 04:44:19.422547 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.422587 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.424020 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.424521 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.424527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.424609 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:19.425294 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.425310 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.426350 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.426355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.427419 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.427432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.427949 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.427952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.428098 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.428687 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:19.428689 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:19.428729 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:19.428792 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:19.429053 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:19.429060 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:19.429063 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.429173 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.429472 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":101,"threads_started":1}
I20260809 04:44:19.431408 28531 heartbeater.cc:347] Connected to a master server at 127.5.222.254:46151
I20260809 04:44:19.431643 28531 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:19.432137 28531 heartbeater.cc:511] Master 127.5.222.254:46151 requested a full tablet report, sending...
I20260809 04:44:19.433115 28052 ts_manager.cc:194] Registered new tserver with Master: 2aa5624e70bc4b0983258bf6290f2f6e (127.5.222.195:34597)
I20260809 04:44:19.433732 28052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.222.195:55521
I20260809 04:44:19.439225 6011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:44:19.444877 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.441276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57810 (local address 127.5.222.254:46151)
0809 04:44:19.441425 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.441430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.441648 (+ 218us) server_negotiation.cc:407] Connection header received
0809 04:44:19.441764 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.441767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.441809 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.441881 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:19.442160 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.442662 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.443437 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.443638 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.443716 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.444436 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:19.444439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:19.444441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:19.444452 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:19.444469 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:19.444477 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:19.444535 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.444606 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.444648 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:44:19.447510 28052 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:19.450672 28052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:19.464778 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.459235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33832 (local address 127.5.222.195:34597)
0809 04:44:19.459760 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.459777 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.460664 (+ 887us) server_negotiation.cc:407] Connection header received
0809 04:44:19.460771 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.460776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.460936 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.461102 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:19.461351 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.462103 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.463558 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.464093 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.464215 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.464412 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.464522 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.464615 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":158,"threads_started":1}
I20260809 04:44:19.464773 28547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.458979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:34597 (local address 127.0.0.1:33832)
0809 04:44:19.460536 (+ 1557us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.460550 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.460657 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.461083 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.461087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.461105 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:19.461234 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.461242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.462222 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.462227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.463439 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.463449 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.464228 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.464232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.464341 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.464376 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.464495 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.464617 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1496,"mutex_wait_us":771,"thread_start_us":114,"threads_started":1}
I20260809 04:44:19.465787 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.458979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.194:44561 (local address 127.0.0.1:35172)
0809 04:44:19.460038 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.460090 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.460230 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.461007 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.461011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.461030 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:19.461145 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.461174 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.462591 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.462595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.464267 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.464279 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.465391 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.465395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.465494 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.465520 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.465591 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.465655 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":166,"spinlock_wait_cycles":9472,"thread_start_us":98,"threads_started":1}
I20260809 04:44:19.465899 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.459129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35172 (local address 127.5.222.194:44561)
0809 04:44:19.459896 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.459910 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.460380 (+ 470us) server_negotiation.cc:407] Connection header received
0809 04:44:19.460471 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.460477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.460677 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.460846 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:19.461319 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.462452 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.464428 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.465259 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.465384 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.465557 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.465617 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.465715 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":166,"threads_started":1}
I20260809 04:44:19.466329 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.459174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:42059 (local address 127.0.0.1:50300)
0809 04:44:19.460051 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.460095 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.460231 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.460815 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.460821 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.460844 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:19.460994 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.461002 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.462592 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.462595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.464268 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.464279 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.465949 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.465952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.466034 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.466057 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.466133 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.466207 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":140,"spinlock_wait_cycles":768,"thread_start_us":113,"threads_started":1}
I20260809 04:44:19.466482 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.459838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50300 (local address 127.5.222.193:42059)
0809 04:44:19.460299 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.460312 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.460336 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:44:19.460415 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.460420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.460635 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.460793 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:19.461145 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.462428 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.464428 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.465262 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.465383 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.465557 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.466180 (+ 623us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.466293 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":163,"threads_started":1}
I20260809 04:44:19.468411 28320 tablet_service.cc:1467] Processing CreateTablet for tablet 39513bb3b1d044e79f73862d8cad06a7 (DEFAULT_TABLE table=test-table [id=864bdfa4b2d147049b20ae59e964fa95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:19.468447 28461 tablet_service.cc:1467] Processing CreateTablet for tablet 39513bb3b1d044e79f73862d8cad06a7 (DEFAULT_TABLE table=test-table [id=864bdfa4b2d147049b20ae59e964fa95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:19.468892 28181 tablet_service.cc:1467] Processing CreateTablet for tablet 39513bb3b1d044e79f73862d8cad06a7 (DEFAULT_TABLE table=test-table [id=864bdfa4b2d147049b20ae59e964fa95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:19.469499 28320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39513bb3b1d044e79f73862d8cad06a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:19.469609 28461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39513bb3b1d044e79f73862d8cad06a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:19.469964 28181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39513bb3b1d044e79f73862d8cad06a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:19.476603 28549 tablet_bootstrap.cc:492] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c: Bootstrap starting.
I20260809 04:44:19.477504 28550 tablet_bootstrap.cc:492] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4: Bootstrap starting.
I20260809 04:44:19.479128 28549 tablet_bootstrap.cc:654] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:19.479730 28550 tablet_bootstrap.cc:654] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:19.480063 28549 log.cc:824] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:19.480576 28550 log.cc:824] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:19.480875 28551 tablet_bootstrap.cc:492] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e: Bootstrap starting.
I20260809 04:44:19.482419 28549 tablet_bootstrap.cc:492] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c: No bootstrap required, opened a new log
I20260809 04:44:19.482635 28549 ts_tablet_manager.cc:1397] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:44:19.482771 28550 tablet_bootstrap.cc:492] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4: No bootstrap required, opened a new log
I20260809 04:44:19.482986 28550 ts_tablet_manager.cc:1397] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:44:19.483268 28551 tablet_bootstrap.cc:654] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:19.483946 28551 log.cc:824] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:19.485673 28551 tablet_bootstrap.cc:492] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e: No bootstrap required, opened a new log
I20260809 04:44:19.485880 28551 ts_tablet_manager.cc:1397] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:44:19.487162 28550 raft_consensus.cc:348] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d03632453ada497198778ee86169276c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44561 } } peers { permanent_uuid: "2aa5624e70bc4b0983258bf6290f2f6e" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34597 } } peers { permanent_uuid: "40dc05aa2cc741b5bc52da187f76abd4" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42059 } }
I20260809 04:44:19.487161 28549 raft_consensus.cc:348] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d03632453ada497198778ee86169276c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44561 } } peers { permanent_uuid: "2aa5624e70bc4b0983258bf6290f2f6e" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34597 } } peers { permanent_uuid: "40dc05aa2cc741b5bc52da187f76abd4" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42059 } }
I20260809 04:44:19.487437 28550 raft_consensus.cc:374] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:19.487449 28549 raft_consensus.cc:374] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:19.487502 28549 raft_consensus.cc:729] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d03632453ada497198778ee86169276c, State: Initialized, Role: FOLLOWER
I20260809 04:44:19.487502 28550 raft_consensus.cc:729] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40dc05aa2cc741b5bc52da187f76abd4, State: Initialized, Role: FOLLOWER
I20260809 04:44:19.487967 28549 consensus_queue.cc:260] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d03632453ada497198778ee86169276c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44561 } } peers { permanent_uuid: "2aa5624e70bc4b0983258bf6290f2f6e" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34597 } } peers { permanent_uuid: "40dc05aa2cc741b5bc52da187f76abd4" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42059 } }
I20260809 04:44:19.487967 28550 consensus_queue.cc:260] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d03632453ada497198778ee86169276c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44561 } } peers { permanent_uuid: "2aa5624e70bc4b0983258bf6290f2f6e" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34597 } } peers { permanent_uuid: "40dc05aa2cc741b5bc52da187f76abd4" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42059 } }
I20260809 04:44:19.488992 28549 ts_tablet_manager.cc:1428] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 04:44:19.488993 28550 ts_tablet_manager.cc:1428] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 04:44:19.489379 28391 heartbeater.cc:503] Master 127.5.222.254:46151 was elected leader, sending a full tablet report...
I20260809 04:44:19.489379 28252 heartbeater.cc:503] Master 127.5.222.254:46151 was elected leader, sending a full tablet report...
I20260809 04:44:19.489774 28551 raft_consensus.cc:348] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d03632453ada497198778ee86169276c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44561 } } peers { permanent_uuid: "2aa5624e70bc4b0983258bf6290f2f6e" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34597 } } peers { permanent_uuid: "40dc05aa2cc741b5bc52da187f76abd4" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42059 } }
I20260809 04:44:19.490018 28551 raft_consensus.cc:374] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:19.490098 28551 raft_consensus.cc:729] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aa5624e70bc4b0983258bf6290f2f6e, State: Initialized, Role: FOLLOWER
I20260809 04:44:19.490619 28551 consensus_queue.cc:260] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d03632453ada497198778ee86169276c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44561 } } peers { permanent_uuid: "2aa5624e70bc4b0983258bf6290f2f6e" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34597 } } peers { permanent_uuid: "40dc05aa2cc741b5bc52da187f76abd4" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42059 } }
I20260809 04:44:19.491418 28531 heartbeater.cc:503] Master 127.5.222.254:46151 was elected leader, sending a full tablet report...
I20260809 04:44:19.491737 28551 ts_tablet_manager.cc:1428] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:44:19.536006 28557 raft_consensus.cc:482] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:19.536273 28557 raft_consensus.cc:504] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d03632453ada497198778ee86169276c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44561 } } peers { permanent_uuid: "2aa5624e70bc4b0983258bf6290f2f6e" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34597 } } peers { permanent_uuid: "40dc05aa2cc741b5bc52da187f76abd4" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42059 } }
I20260809 04:44:19.537758 28557 leader_election.cc:288] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2aa5624e70bc4b0983258bf6290f2f6e (127.5.222.195:34597), 40dc05aa2cc741b5bc52da187f76abd4 (127.5.222.193:42059)
I20260809 04:44:19.541998 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.538058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:38985 (local address 127.5.222.195:34597)
0809 04:44:19.538179 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.538184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.538303 (+ 119us) server_negotiation.cc:407] Connection header received
0809 04:44:19.538383 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.538387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.538447 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.538530 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:19.538846 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.539481 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.540726 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.541346 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.541466 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.541533 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.541791 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.541868 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:44:19.541998 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.538137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.222.194:48817 (local address 127.5.222.193:42059)
0809 04:44:19.538300 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.538305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.538469 (+ 164us) server_negotiation.cc:407] Connection header received
0809 04:44:19.538611 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.538614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.538663 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.538752 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:19.539046 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.539552 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.540827 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.541439 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.541546 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.541595 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.541790 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.541893 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 04:44:19.542014 28393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.537932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.195:34597 (local address 127.5.222.194:38985)
0809 04:44:19.538166 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.538181 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.538280 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.538567 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.538570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.538590 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:19.538728 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.538739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.539597 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.539603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.540593 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.540603 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.541479 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.541482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.541610 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.541673 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.541758 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.541864 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":158}
I20260809 04:44:19.542022 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.538027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:42059 (local address 127.5.222.194:48817)
0809 04:44:19.538372 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.538385 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.538503 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.538768 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.538771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.538805 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:19.538930 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.538936 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.539670 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.539674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.540601 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.540614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.541545 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.541548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.541676 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.541692 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.541759 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.541855 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":34,"thread_start_us":108,"threads_started":1}
W20260809 04:44:19.542497 28253 tablet.cc:2369] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:19.542786 28483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39513bb3b1d044e79f73862d8cad06a7" candidate_uuid: "d03632453ada497198778ee86169276c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2aa5624e70bc4b0983258bf6290f2f6e" is_pre_election: true
I20260809 04:44:19.542842 28202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39513bb3b1d044e79f73862d8cad06a7" candidate_uuid: "d03632453ada497198778ee86169276c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40dc05aa2cc741b5bc52da187f76abd4" is_pre_election: true
I20260809 04:44:19.543118 28483 raft_consensus.cc:2454] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d03632453ada497198778ee86169276c in term 0.
I20260809 04:44:19.543185 28202 raft_consensus.cc:2454] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d03632453ada497198778ee86169276c in term 0.
I20260809 04:44:19.543675 28271 leader_election.cc:302] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aa5624e70bc4b0983258bf6290f2f6e, d03632453ada497198778ee86169276c; no voters:
I20260809 04:44:19.543948 28557 raft_consensus.cc:2781] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:19.544046 28557 raft_consensus.cc:482] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:19.544106 28557 raft_consensus.cc:3037] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:19.545295 28557 raft_consensus.cc:504] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d03632453ada497198778ee86169276c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44561 } } peers { permanent_uuid: "2aa5624e70bc4b0983258bf6290f2f6e" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34597 } } peers { permanent_uuid: "40dc05aa2cc741b5bc52da187f76abd4" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42059 } }
I20260809 04:44:19.545661 28557 leader_election.cc:288] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [CANDIDATE]: Term 1 election: Requested vote from peers 2aa5624e70bc4b0983258bf6290f2f6e (127.5.222.195:34597), 40dc05aa2cc741b5bc52da187f76abd4 (127.5.222.193:42059)
I20260809 04:44:19.546056 28202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39513bb3b1d044e79f73862d8cad06a7" candidate_uuid: "d03632453ada497198778ee86169276c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40dc05aa2cc741b5bc52da187f76abd4"
I20260809 04:44:19.546062 28483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39513bb3b1d044e79f73862d8cad06a7" candidate_uuid: "d03632453ada497198778ee86169276c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2aa5624e70bc4b0983258bf6290f2f6e"
I20260809 04:44:19.546187 28202 raft_consensus.cc:3037] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:19.546187 28483 raft_consensus.cc:3037] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:19.547489 28483 raft_consensus.cc:2454] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d03632453ada497198778ee86169276c in term 1.
I20260809 04:44:19.547490 28202 raft_consensus.cc:2454] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d03632453ada497198778ee86169276c in term 1.
I20260809 04:44:19.547804 28273 leader_election.cc:302] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40dc05aa2cc741b5bc52da187f76abd4, d03632453ada497198778ee86169276c; no voters:
I20260809 04:44:19.548000 28557 raft_consensus.cc:2781] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:19.548234 28557 raft_consensus.cc:686] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 1 LEADER]: Becoming Leader. State: Replica: d03632453ada497198778ee86169276c, State: Running, Role: LEADER
I20260809 04:44:19.548564 28557 consensus_queue.cc:237] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d03632453ada497198778ee86169276c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44561 } } peers { permanent_uuid: "2aa5624e70bc4b0983258bf6290f2f6e" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34597 } } peers { permanent_uuid: "40dc05aa2cc741b5bc52da187f76abd4" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42059 } }
I20260809 04:44:19.551975 28051 catalog_manager.cc:5495] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c reported cstate change: term changed from 0 to 1, leader changed from <none> to d03632453ada497198778ee86169276c (127.5.222.194). New cstate: current_term: 1 leader_uuid: "d03632453ada497198778ee86169276c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d03632453ada497198778ee86169276c" member_type: VOTER last_known_addr { host: "127.5.222.194" port: 44561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2aa5624e70bc4b0983258bf6290f2f6e" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40dc05aa2cc741b5bc52da187f76abd4" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42059 } health_report { overall_health: UNKNOWN } } }
I20260809 04:44:19.564620 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.561269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35176 (local address 127.5.222.194:44561)
0809 04:44:19.561414 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.561418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.561440 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:44:19.561514 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.561516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.561574 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.561672 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:19.561935 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.562464 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.563306 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.563517 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.563600 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.563862 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:19.564294 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:19.564388 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.564445 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.564491 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:44:19.570758 28202 raft_consensus.cc:1261] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4 [term 1 FOLLOWER]: Refusing update from remote peer d03632453ada497198778ee86169276c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:19.570758 28483 raft_consensus.cc:1261] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e [term 1 FOLLOWER]: Refusing update from remote peer d03632453ada497198778ee86169276c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:19.571555 28560 consensus_queue.cc:1035] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2aa5624e70bc4b0983258bf6290f2f6e" member_type: VOTER last_known_addr { host: "127.5.222.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:19.571771 28557 consensus_queue.cc:1035] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40dc05aa2cc741b5bc52da187f76abd4" member_type: VOTER last_known_addr { host: "127.5.222.193" port: 42059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:19.579795 28564 mvcc.cc:205] Tried to move back new op lower bound from 7316482701593702400 to 7316482701513879552. Current Snapshot: MvccSnapshot[applied={T|T < 7316482701593702400}]
I20260809 04:44:19.588672 28052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:19.588860 28052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:19.591601 28052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 992983cf7a944d5d97c8f55dc23ed17e: Sending DeleteTablet for 3 replicas of tablet 39513bb3b1d044e79f73862d8cad06a7
I20260809 04:44:19.592537 28320 tablet_service.cc:1514] Processing DeleteTablet for tablet 39513bb3b1d044e79f73862d8cad06a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:19 UTC) from {username='slave'} at 127.0.0.1:35172
I20260809 04:44:19.592490 28461 tablet_service.cc:1514] Processing DeleteTablet for tablet 39513bb3b1d044e79f73862d8cad06a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:19 UTC) from {username='slave'} at 127.0.0.1:33832
I20260809 04:44:19.592614 28181 tablet_service.cc:1514] Processing DeleteTablet for tablet 39513bb3b1d044e79f73862d8cad06a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:19 UTC) from {username='slave'} at 127.0.0.1:50300
I20260809 04:44:19.593066 28574 tablet_replica.cc:330] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e: stopping tablet replica
I20260809 04:44:19.593184 28573 tablet_replica.cc:330] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c: stopping tablet replica
I20260809 04:44:19.593448 28574 raft_consensus.cc:2229] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:19.593494 28573 raft_consensus.cc:2229] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:19.593904 28573 raft_consensus.cc:2258] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:19.594031 28575 tablet_replica.cc:330] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4: stopping tablet replica
I20260809 04:44:19.594300 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 28116
I20260809 04:44:19.594420 28575 raft_consensus.cc:2229] T 39513bb3b1d044e79f73862d8cad06a7 P 40dc05aa2cc741b5bc52da187f76abd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:19.595599 28573 ts_tablet_manager.cc:1905] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:19.599016 28573 ts_tablet_manager.cc:1918] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:19.599143 28573 log.cc:1196] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-1/wal/wals/39513bb3b1d044e79f73862d8cad06a7
I20260809 04:44:19.599514 28573 ts_tablet_manager.cc:1939] T 39513bb3b1d044e79f73862d8cad06a7 P d03632453ada497198778ee86169276c: Deleting consensus metadata
I20260809 04:44:19.601480 28037 catalog_manager.cc:4841] TS d03632453ada497198778ee86169276c (127.5.222.194:44561): tablet 39513bb3b1d044e79f73862d8cad06a7 (table test-table [id=864bdfa4b2d147049b20ae59e964fa95]) successfully deleted
I20260809 04:44:19.593789 28574 raft_consensus.cc:2258] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:19.603471 28574 ts_tablet_manager.cc:1905] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:19.605809 28574 ts_tablet_manager.cc:1918] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:19.605939 28574 log.cc:1196] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e: Deleting WAL directory at /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250403492198-6011-0/minicluster-data/ts-2/wal/wals/39513bb3b1d044e79f73862d8cad06a7
I20260809 04:44:19.606241 28574 ts_tablet_manager.cc:1939] T 39513bb3b1d044e79f73862d8cad06a7 P 2aa5624e70bc4b0983258bf6290f2f6e: Deleting consensus metadata
I20260809 04:44:19.607618 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 28255
W20260809 04:44:19.607630 28038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.222.193:42059 (error 108)
I20260809 04:44:19.607729 28036 catalog_manager.cc:4841] TS 2aa5624e70bc4b0983258bf6290f2f6e (127.5.222.195:34597): tablet 39513bb3b1d044e79f73862d8cad06a7 (table test-table [id=864bdfa4b2d147049b20ae59e964fa95]) successfully deleted
I20260809 04:44:19.610970 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.610562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.193:42059 (local address 127.0.0.1:50308)
0809 04:44:19.610761 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.610855 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.193:42059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 04:44:19.611424 28038 catalog_manager.cc:4584] TS 40dc05aa2cc741b5bc52da187f76abd4 (127.5.222.193:42059): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 39513bb3b1d044e79f73862d8cad06a7: Network error: Client connection negotiation failed: client connection to 127.5.222.193:42059: connect: Connection refused (error 111)
I20260809 04:44:19.616681 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 28394
I20260809 04:44:19.623824 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 28019
2026-08-09T04:44:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1032 ms)
[----------] 6 tests from EncryptionPolicyTest (10819 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:19 dist-test-slave-vxj2 krb5kdc[28579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:19 dist-test-slave-vxj2 krb5kdc[28579](info): set up 2 sockets
Aug 09 04:44:19 dist-test-slave-vxj2 krb5kdc[28579](info): commencing operation
krb5kdc: starting...
W20260809 04:44:20.734349 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:20 dist-test-slave-vxj2 krb5kdc[28579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:44:20.864750 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41621
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:20.977824 28592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:20.978089 28592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:20.978130 28592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:20.982133 28592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:20.982201 28592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:20.982221 28592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:20.982268 28592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:20.982301 28592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:20.987656 28592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41621
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:20.988796 28592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:20.989902 28592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:20.998373 28601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:20.998349 28599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:20.998349 28598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:20.998934 28592 server_base.cc:1034] running on GCE node
I20260809 04:44:20.999442 28592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:21.000587 28592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:21.001802 28592 hybrid_clock.cc:648] HybridClock initialized: now 1786250661001776 us; error 45 us; skew 500 ppm
Aug 09 04:44:21 dist-test-slave-vxj2 krb5kdc[28579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250661, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:21.004727 28592 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:44:21.005663 28592 webserver.cc:460] Webserver started at http://127.0.0.1:35371/ using document root <none> and password file <none>
I20260809 04:44:21.006193 28592 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:21.006251 28592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:21.006414 28592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:21.008451 28592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "4c3bff2e94bc402893b4754b606e4a52"
format_stamp: "Formatted at 2026-08-09 04:44:21 on dist-test-slave-vxj2"
server_key: "fd422411c020b23b2d7f62e515efd040"
server_key_iv: "d2dde458de4d791854a2a1a0ab4a7ffa"
server_key_version: "encryptionkey@0"
I20260809 04:44:21.008998 28592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "4c3bff2e94bc402893b4754b606e4a52"
format_stamp: "Formatted at 2026-08-09 04:44:21 on dist-test-slave-vxj2"
server_key: "fd422411c020b23b2d7f62e515efd040"
server_key_iv: "d2dde458de4d791854a2a1a0ab4a7ffa"
server_key_version: "encryptionkey@0"
I20260809 04:44:21.012962 28592 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260809 04:44:21.015595 28609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:21.016901 28592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:44:21.017058 28592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "4c3bff2e94bc402893b4754b606e4a52"
format_stamp: "Formatted at 2026-08-09 04:44:21 on dist-test-slave-vxj2"
server_key: "fd422411c020b23b2d7f62e515efd040"
server_key_iv: "d2dde458de4d791854a2a1a0ab4a7ffa"
server_key_version: "encryptionkey@0"
I20260809 04:44:21.017237 28592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:21.035622 28592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:21.039076 28592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:21.039306 28592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:21.046722 28592 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41621
I20260809 04:44:21.046798 28664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41621 every 8 connection(s)
I20260809 04:44:21.047751 28592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:21.050832 28665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:21.051548 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 28592
I20260809 04:44:21.051700 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:21.052011 6011 external_mini_cluster.cc:1428] Setting key d7680e3bea0a9811075548cf3fc5fa6a
I20260809 04:44:21.057834 28665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52: Bootstrap starting.
I20260809 04:44:21.060433 28665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:21.061242 28665 log.cc:824] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:44:21 dist-test-slave-vxj2 krb5kdc[28579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 04:44:21.063560 28665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52: No bootstrap required, opened a new log
I20260809 04:44:21.067847 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:21.053526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53325 (local address 127.0.0.1:41621)
0809 04:44:21.053981 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:21.054002 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:21.054046 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:44:21.054870 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:21.054938 (+ 68us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:21.055366 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:21.055731 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:21.056524 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.058023 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.058886 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.059583 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.059737 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:21.062619 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:21.062625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:21.062640 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:21.062670 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:21.064587 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:21.065069 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:21.065075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:21.065081 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:21.065240 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:21.065584 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:21.065589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:21.065591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:21.066179 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:21.066388 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:21.066678 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 04:44:21.066906 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":108,"threads_started":1}
I20260809 04:44:21.068259 28665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3bff2e94bc402893b4754b606e4a52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41621 } }
I20260809 04:44:21.068529 28665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:21.068593 28665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c3bff2e94bc402893b4754b606e4a52, State: Initialized, Role: FOLLOWER
I20260809 04:44:21.069046 28665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3bff2e94bc402893b4754b606e4a52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41621 } }
I20260809 04:44:21.069219 28665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:21.069291 28665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:21.069391 28665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:21.070519 28665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3bff2e94bc402893b4754b606e4a52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41621 } }
I20260809 04:44:21.070876 28665 leader_election.cc:302] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c3bff2e94bc402893b4754b606e4a52; no voters:
I20260809 04:44:21.070993 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:21.071230 28665 leader_election.cc:288] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:21.071360 28671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:21.071645 28671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [term 1 LEADER]: Becoming Leader. State: Replica: 4c3bff2e94bc402893b4754b606e4a52, State: Running, Role: LEADER
I20260809 04:44:21.072000 28671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3bff2e94bc402893b4754b606e4a52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41621 } }
I20260809 04:44:21.072561 28665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:21.073894 28673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c3bff2e94bc402893b4754b606e4a52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3bff2e94bc402893b4754b606e4a52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41621 } } }
I20260809 04:44:21.074023 28673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:21.074244 28673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c3bff2e94bc402893b4754b606e4a52. Latest consensus state: current_term: 1 leader_uuid: "4c3bff2e94bc402893b4754b606e4a52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3bff2e94bc402893b4754b606e4a52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41621 } } }
I20260809 04:44:21.074316 28673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52 [sys.catalog]: This master's current role is: LEADER
W20260809 04:44:21.077910 28692 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:21.078047 28692 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:21.078191 28687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:21.079071 28687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:21.082558 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:21.073203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47467 (local address 127.0.0.1:41621)
0809 04:44:21.073308 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:21.073313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:21.073604 (+ 291us) server_negotiation.cc:407] Connection header received
0809 04:44:21.073798 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:21.073801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:21.073846 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:21.073908 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:21.074934 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.075724 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.076545 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.076781 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.076975 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:21.078521 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:21.078525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:21.078528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:21.078553 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:21.080225 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:21.080827 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:21.080831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:21.080833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:21.080921 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:21.081462 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:21.081467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:21.081469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:21.081706 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:21.081829 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:21.082285 (+ 456us) server_negotiation.cc:299] Negotiation successful
0809 04:44:21.082392 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 04:44:21.084465 28687 catalog_manager.cc:1347] Generated new cluster ID: 1b7173f63fe241f9bf03e330be423c6e
I20260809 04:44:21.084550 28687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:21.094709 28687 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:21.095602 28687 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:21.100507 28687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c3bff2e94bc402893b4754b606e4a52: Generated new TSK 0
I20260809 04:44:21.101235 28687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:21.108662 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 28592
W20260809 04:44:21.116145 28577 connection.cc:535] client connection to 127.0.0.1:41621 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41621): Connection reset by peer (error 104)
2026-08-09T04:44:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1481 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:21 dist-test-slave-vxj2 krb5kdc[28698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:21 dist-test-slave-vxj2 krb5kdc[28698](info): set up 2 sockets
Aug 09 04:44:21 dist-test-slave-vxj2 krb5kdc[28698](info): commencing operation
krb5kdc: starting...
W20260809 04:44:22.213315 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:22 dist-test-slave-vxj2 krb5kdc[28698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:44:22.339983 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41467
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:22.456295 28711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:22.456558 28711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:22.456604 28711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:22.460788 28711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:22.460858 28711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:22.460878 28711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:22.460896 28711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:22.460911 28711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:22.466765 28711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41467
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:22.467760 28711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:22.468768 28711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:22.477558 28721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:22.477589 28717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:22.477576 28716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:22.477792 28711 server_base.cc:1034] running on GCE node
I20260809 04:44:22.478489 28711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:22.479537 28711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:22.480723 28711 hybrid_clock.cc:648] HybridClock initialized: now 1786250662480699 us; error 40 us; skew 500 ppm
Aug 09 04:44:22 dist-test-slave-vxj2 krb5kdc[28698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250662, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:22.483578 28711 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:44:22.484488 28711 webserver.cc:460] Webserver started at http://127.0.0.1:41635/ using document root <none> and password file <none>
I20260809 04:44:22.485042 28711 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:22.485136 28711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:22.485385 28711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:22.487498 28711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "cb2faf8690a24063a186e926539a1919"
format_stamp: "Formatted at 2026-08-09 04:44:22 on dist-test-slave-vxj2"
server_key: "11467fe2efa5c253562514ae199ef799"
server_key_iv: "90c2db329d32c6e1b4370b4c74dd4542"
server_key_version: "encryptionkey@0"
I20260809 04:44:22.488078 28711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "cb2faf8690a24063a186e926539a1919"
format_stamp: "Formatted at 2026-08-09 04:44:22 on dist-test-slave-vxj2"
server_key: "11467fe2efa5c253562514ae199ef799"
server_key_iv: "90c2db329d32c6e1b4370b4c74dd4542"
server_key_version: "encryptionkey@0"
I20260809 04:44:22.491869 28711 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 04:44:22.494320 28728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:22.495476 28711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:44:22.495615 28711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "cb2faf8690a24063a186e926539a1919"
format_stamp: "Formatted at 2026-08-09 04:44:22 on dist-test-slave-vxj2"
server_key: "11467fe2efa5c253562514ae199ef799"
server_key_iv: "90c2db329d32c6e1b4370b4c74dd4542"
server_key_version: "encryptionkey@0"
I20260809 04:44:22.495726 28711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:22.509105 28711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:22.511927 28711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:22.512130 28711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:22.519929 28711 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41467
I20260809 04:44:22.519954 28784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41467 every 8 connection(s)
I20260809 04:44:22.520861 28711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:22.523558 28785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:22.526815 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 28711
I20260809 04:44:22.526925 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:22.527251 6011 external_mini_cluster.cc:1428] Setting key 3b6c55c8c58fe8797c0f3e8433b4ddb3
I20260809 04:44:22.529722 28785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919: Bootstrap starting.
I20260809 04:44:22.532475 28785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:22.533349 28785 log.cc:824] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:22.535494 28785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919: No bootstrap required, opened a new log
I20260809 04:44:22.537624 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.528802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39563 (local address 127.0.0.1:41467)
0809 04:44:22.529271 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.529312 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.529354 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:44:22.530179 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.530202 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.530624 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.531015 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:22.531434 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.532918 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.534006 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.534661 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.534824 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.535689 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:22.535694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:22.535706 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:22.536091 (+ 385us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:22.536147 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:22.536170 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:22.536316 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.536503 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.536643 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":128,"threads_started":1}
I20260809 04:44:22.538739 28785 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2faf8690a24063a186e926539a1919" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41467 } }
I20260809 04:44:22.539004 28785 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:22.539054 28785 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb2faf8690a24063a186e926539a1919, State: Initialized, Role: FOLLOWER
I20260809 04:44:22.539521 28785 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2faf8690a24063a186e926539a1919" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41467 } }
I20260809 04:44:22.539714 28785 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:22.539805 28785 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:22.539912 28785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:22.540676 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:22.541008 28785 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2faf8690a24063a186e926539a1919" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41467 } }
I20260809 04:44:22.541363 28785 leader_election.cc:302] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb2faf8690a24063a186e926539a1919; no voters:
I20260809 04:44:22.541690 28785 leader_election.cc:288] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:22.541760 28790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:22.541990 28790 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [term 1 LEADER]: Becoming Leader. State: Replica: cb2faf8690a24063a186e926539a1919, State: Running, Role: LEADER
I20260809 04:44:22.542337 28790 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2faf8690a24063a186e926539a1919" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41467 } }
I20260809 04:44:22.542963 28785 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:22.543594 28793 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb2faf8690a24063a186e926539a1919. Latest consensus state: current_term: 1 leader_uuid: "cb2faf8690a24063a186e926539a1919" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2faf8690a24063a186e926539a1919" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41467 } } }
I20260809 04:44:22.543792 28793 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:22.543990 28791 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb2faf8690a24063a186e926539a1919" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2faf8690a24063a186e926539a1919" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41467 } } }
I20260809 04:44:22.544095 28791 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:22.544871 28806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:22.548043 28806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:22.548686 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.542984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40215 (local address 127.0.0.1:41467)
0809 04:44:22.543100 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.543104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.544295 (+ 1191us) server_negotiation.cc:407] Connection header received
0809 04:44:22.544483 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.544486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.544535 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.544617 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:22.545040 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.545793 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.546907 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.547149 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.547263 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.548147 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:22.548150 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:22.548153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:22.548166 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:22.548198 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:22.548209 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:22.548308 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.548502 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.548544 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 04:44:22.554118 28806 catalog_manager.cc:1347] Generated new cluster ID: a662d17893124fac8b2511f4c4bf9bfd
I20260809 04:44:22.554188 28806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:22.611281 28806 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:22.612272 28806 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:22.619318 28806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb2faf8690a24063a186e926539a1919: Generated new TSK 0
I20260809 04:44:22.620014 28806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:22.635116 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 28711
W20260809 04:44:22.642640 28696 connection.cc:535] client connection to 127.0.0.1:41467 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41467): Connection reset by peer (error 104)
2026-08-09T04:44:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1526 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:22 dist-test-slave-vxj2 krb5kdc[28819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:22 dist-test-slave-vxj2 krb5kdc[28819](info): set up 2 sockets
Aug 09 04:44:22 dist-test-slave-vxj2 krb5kdc[28819](info): commencing operation
krb5kdc: starting...
W20260809 04:44:23.733893 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:23 dist-test-slave-vxj2 krb5kdc[28819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:44:23.860775 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44511
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:23.969400 28835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:23.969647 28835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:23.969688 28835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:23.973519 28835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:23.973570 28835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:23.973589 28835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:23.973606 28835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:23.973621 28835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:23.978664 28835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44511
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:23.979593 28835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:23.980501 28835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:23.988790 28841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:23.988715 28840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:23.988894 28843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:23.989110 28835 server_base.cc:1034] running on GCE node
I20260809 04:44:23.989672 28835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:23.990691 28835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:23.991907 28835 hybrid_clock.cc:648] HybridClock initialized: now 1786250663991888 us; error 30 us; skew 500 ppm
Aug 09 04:44:23 dist-test-slave-vxj2 krb5kdc[28819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:23.994683 28835 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:44:23.995496 28835 webserver.cc:460] Webserver started at http://127.0.0.1:38933/ using document root <none> and password file <none>
I20260809 04:44:23.996003 28835 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:23.996060 28835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:23.996263 28835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:23.998365 28835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "518dd9fd00cb4b838e5c43cff580f345"
format_stamp: "Formatted at 2026-08-09 04:44:23 on dist-test-slave-vxj2"
server_key: "0b59fcb6ac6f2c95bdb83187f3b9dbe4"
server_key_iv: "36bf1a4b14fbd7d7f034cd4a615d0647"
server_key_version: "encryptionkey@0"
I20260809 04:44:23.998926 28835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "518dd9fd00cb4b838e5c43cff580f345"
format_stamp: "Formatted at 2026-08-09 04:44:23 on dist-test-slave-vxj2"
server_key: "0b59fcb6ac6f2c95bdb83187f3b9dbe4"
server_key_iv: "36bf1a4b14fbd7d7f034cd4a615d0647"
server_key_version: "encryptionkey@0"
I20260809 04:44:24.002645 28835 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:44:24.005276 28850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:24.006496 28835 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:24.006636 28835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "518dd9fd00cb4b838e5c43cff580f345"
format_stamp: "Formatted at 2026-08-09 04:44:23 on dist-test-slave-vxj2"
server_key: "0b59fcb6ac6f2c95bdb83187f3b9dbe4"
server_key_iv: "36bf1a4b14fbd7d7f034cd4a615d0647"
server_key_version: "encryptionkey@0"
I20260809 04:44:24.006744 28835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:24.011464 28835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:24.014621 28835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:24.014822 28835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:24.022298 28835 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44511
I20260809 04:44:24.022328 28907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44511 every 8 connection(s)
I20260809 04:44:24.023240 28835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:24.026036 28908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:24.026762 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 28835
I20260809 04:44:24.026919 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:24.027235 6011 external_mini_cluster.cc:1428] Setting key 2173d69c864506bf97921badd993f1ce
I20260809 04:44:24.032534 28908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345: Bootstrap starting.
I20260809 04:44:24.034855 28908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:24.035624 28908 log.cc:824] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:24.035835 28911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.028685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60575 (local address 127.0.0.1:44511)
0809 04:44:24.029130 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:24.029144 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:24.029242 (+ 98us) server_negotiation.cc:407] Connection header received
0809 04:44:24.029887 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:24.029903 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:24.030181 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:24.030531 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:24.034504 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:24.034508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:24.034522 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:24.034533 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:24.034589 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:24.034608 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:24.034734 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:24.034874 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 04:44:24.035042 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":135,"threads_started":1}
I20260809 04:44:24.037874 28908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345: No bootstrap required, opened a new log
I20260809 04:44:24.038815 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:24.042140 28908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518dd9fd00cb4b838e5c43cff580f345" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44511 } }
I20260809 04:44:24.042451 28908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:24.042527 28908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 518dd9fd00cb4b838e5c43cff580f345, State: Initialized, Role: FOLLOWER
I20260809 04:44:24.042569 28911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.040862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34815 (local address 127.0.0.1:44511)
0809 04:44:24.040979 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:24.040986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:24.041202 (+ 216us) server_negotiation.cc:407] Connection header received
0809 04:44:24.041332 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:24.041334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:24.041363 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:24.041429 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:24.042122 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:24.042125 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:24.042127 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:24.042136 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:24.042159 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:24.042168 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:24.042235 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:24.042352 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 04:44:24.042383 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:44:24.043030 28908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518dd9fd00cb4b838e5c43cff580f345" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44511 } }
I20260809 04:44:24.043188 28908 raft_consensus.cc:388] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:24.043265 28908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:24.043365 28908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:24.044337 28908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518dd9fd00cb4b838e5c43cff580f345" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44511 } }
I20260809 04:44:24.044618 28908 leader_election.cc:302] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 518dd9fd00cb4b838e5c43cff580f345; no voters:
I20260809 04:44:24.044924 28908 leader_election.cc:288] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:24.045058 28919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:24.045370 28919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [term 1 LEADER]: Becoming Leader. State: Replica: 518dd9fd00cb4b838e5c43cff580f345, State: Running, Role: LEADER
I20260809 04:44:24.045746 28919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518dd9fd00cb4b838e5c43cff580f345" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44511 } }
I20260809 04:44:24.046202 28908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:24.046866 28919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 518dd9fd00cb4b838e5c43cff580f345. Latest consensus state: current_term: 1 leader_uuid: "518dd9fd00cb4b838e5c43cff580f345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518dd9fd00cb4b838e5c43cff580f345" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44511 } } }
I20260809 04:44:24.047084 28919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:24.047300 28920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "518dd9fd00cb4b838e5c43cff580f345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518dd9fd00cb4b838e5c43cff580f345" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44511 } } }
I20260809 04:44:24.047467 28920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:24.047542 28929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:24.050731 28929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:24.055905 28929 catalog_manager.cc:1347] Generated new cluster ID: 6518f58b4ff741c9b7448408d54271d9
I20260809 04:44:24.055979 28929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:24.079625 28929 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:24.079982 28929 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:24.089969 28929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 518dd9fd00cb4b838e5c43cff580f345: Generated new TSK 0
I20260809 04:44:24.090659 28929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:24.095558 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 28835
2026-08-09T04:44:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:24 dist-test-slave-vxj2 krb5kdc[28941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:24 dist-test-slave-vxj2 krb5kdc[28941](info): set up 2 sockets
Aug 09 04:44:24 dist-test-slave-vxj2 krb5kdc[28941](info): commencing operation
krb5kdc: starting...
W20260809 04:44:25.213732 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:25 dist-test-slave-vxj2 krb5kdc[28941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:44:25.340355 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43223
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:25.452066 28954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:25.452312 28954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:25.452353 28954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:25.456161 28954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:25.456210 28954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:25.456230 28954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:25.456247 28954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:25.456262 28954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:25.461308 28954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:25.462211 28954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:25.463126 28954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:25.471606 28960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:25.471653 28962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:25.471675 28959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:25.471961 28954 server_base.cc:1034] running on GCE node
I20260809 04:44:25.472572 28954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:25.473658 28954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:25.474846 28954 hybrid_clock.cc:648] HybridClock initialized: now 1786250665474814 us; error 45 us; skew 500 ppm
Aug 09 04:44:25 dist-test-slave-vxj2 krb5kdc[28941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250665, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:25.477703 28954 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:44:25.478631 28954 webserver.cc:460] Webserver started at http://127.0.0.1:35897/ using document root <none> and password file <none>
I20260809 04:44:25.479179 28954 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:25.479239 28954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:25.479449 28954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:25.481562 28954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "2614e60bbf294a438c433d40ae161eb7"
format_stamp: "Formatted at 2026-08-09 04:44:25 on dist-test-slave-vxj2"
server_key: "408bcbf56da948ef616dce6719633a08"
server_key_iv: "e7992de917548e08841bc265b005fb44"
server_key_version: "encryptionkey@0"
I20260809 04:44:25.482120 28954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "2614e60bbf294a438c433d40ae161eb7"
format_stamp: "Formatted at 2026-08-09 04:44:25 on dist-test-slave-vxj2"
server_key: "408bcbf56da948ef616dce6719633a08"
server_key_iv: "e7992de917548e08841bc265b005fb44"
server_key_version: "encryptionkey@0"
I20260809 04:44:25.485831 28954 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:25.488279 28969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:25.489648 28954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:25.489796 28954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "2614e60bbf294a438c433d40ae161eb7"
format_stamp: "Formatted at 2026-08-09 04:44:25 on dist-test-slave-vxj2"
server_key: "408bcbf56da948ef616dce6719633a08"
server_key_iv: "e7992de917548e08841bc265b005fb44"
server_key_version: "encryptionkey@0"
I20260809 04:44:25.489914 28954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:25.523903 28954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:25.527405 28954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:25.527631 28954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:25.535145 28954 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43223
I20260809 04:44:25.535172 29024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43223 every 8 connection(s)
I20260809 04:44:25.536101 28954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:25.536880 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 28954
I20260809 04:44:25.537020 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:25.537379 6011 external_mini_cluster.cc:1428] Setting key 6aa1e1df478362c54b47e44d33491022
I20260809 04:44:25.539516 29025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:25.545080 29025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7: Bootstrap starting.
Aug 09 04:44:25 dist-test-slave-vxj2 krb5kdc[28941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 04:44:25.547650 29025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:25.548542 29025 log.cc:824] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:25.550755 29025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7: No bootstrap required, opened a new log
I20260809 04:44:25.552017 29028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.538901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55765 (local address 127.0.0.1:43223)
0809 04:44:25.539350 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:25.539393 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:25.539443 (+ 50us) server_negotiation.cc:407] Connection header received
0809 04:44:25.540140 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:25.540156 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:25.540485 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:25.540826 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:25.541687 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:25.542669 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:25.543593 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:25.544066 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:25.544244 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:25.547102 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:25.547107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:25.547120 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:25.547147 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:25.549063 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:25.549624 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:25.549628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:25.549632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:25.549742 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:25.550084 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:25.550085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:25.550086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:25.550459 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:25.550608 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:25.550991 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 04:44:25.551304 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":149,"threads_started":1}
I20260809 04:44:25.554908 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:25.554776 29025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2614e60bbf294a438c433d40ae161eb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43223 } }
I20260809 04:44:25.555044 29025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:25.555099 29025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2614e60bbf294a438c433d40ae161eb7, State: Initialized, Role: FOLLOWER
I20260809 04:44:25.555550 29025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2614e60bbf294a438c433d40ae161eb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43223 } }
I20260809 04:44:25.555706 29025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:25.555783 29025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:25.555873 29025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:25.557032 29025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2614e60bbf294a438c433d40ae161eb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43223 } }
I20260809 04:44:25.557456 29025 leader_election.cc:302] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2614e60bbf294a438c433d40ae161eb7; no voters:
I20260809 04:44:25.557823 29025 leader_election.cc:288] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:25.557911 29035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:25.558161 29035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [term 1 LEADER]: Becoming Leader. State: Replica: 2614e60bbf294a438c433d40ae161eb7, State: Running, Role: LEADER
I20260809 04:44:25.558466 29035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2614e60bbf294a438c433d40ae161eb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43223 } }
I20260809 04:44:25.559216 29025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:25.559515 29035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2614e60bbf294a438c433d40ae161eb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2614e60bbf294a438c433d40ae161eb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43223 } } }
I20260809 04:44:25.559638 29035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:25.559900 29035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2614e60bbf294a438c433d40ae161eb7. Latest consensus state: current_term: 1 leader_uuid: "2614e60bbf294a438c433d40ae161eb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2614e60bbf294a438c433d40ae161eb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43223 } } }
I20260809 04:44:25.560037 29040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:25.560057 29035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:25.563383 29040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:25.566500 29028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.557226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49997 (local address 127.0.0.1:43223)
0809 04:44:25.557341 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:25.557345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:25.557565 (+ 220us) server_negotiation.cc:407] Connection header received
0809 04:44:25.557685 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:25.557688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:25.557737 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:25.557832 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:25.558631 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:25.559410 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:25.560635 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:25.561054 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:25.561172 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:25.562864 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:25.562868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:25.562871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:25.562896 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:25.564568 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:25.565081 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:25.565085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:25.565087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:25.565202 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:25.565527 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:25.565534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:25.565536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:25.565761 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:25.565900 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:25.566196 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 04:44:25.566325 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 04:44:25.569468 29040 catalog_manager.cc:1347] Generated new cluster ID: 157136e0e201469497e7edce923c2756
I20260809 04:44:25.569528 29040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:25.589484 29040 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:25.590785 29040 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:25.597294 29040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2614e60bbf294a438c433d40ae161eb7: Generated new TSK 0
I20260809 04:44:25.598116 29040 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:25.609586 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 28954
2026-08-09T04:44:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:25 dist-test-slave-vxj2 krb5kdc[29061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:25 dist-test-slave-vxj2 krb5kdc[29061](info): set up 2 sockets
Aug 09 04:44:25 dist-test-slave-vxj2 krb5kdc[29061](info): commencing operation
krb5kdc: starting...
W20260809 04:44:26.749514 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:26 dist-test-slave-vxj2 krb5kdc[29061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:44:26.876492 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37171
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:26.987385 29076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:26.987684 29076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:26.987728 29076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:26.991734 29076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:26.991799 29076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:26.991819 29076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:26.991837 29076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:26.991852 29076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:26.997217 29076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:26.998324 29076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:26.999387 29076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:27.008368 29083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:27.008358 29085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:27.008369 29082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:27.008486 29076 server_base.cc:1034] running on GCE node
I20260809 04:44:27.009389 29076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:27.010576 29076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:27.011780 29076 hybrid_clock.cc:648] HybridClock initialized: now 1786250667011749 us; error 58 us; skew 500 ppm
Aug 09 04:44:27 dist-test-slave-vxj2 krb5kdc[29061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250667, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:27.014803 29076 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:44:27.015769 29076 webserver.cc:460] Webserver started at http://127.0.0.1:32885/ using document root <none> and password file <none>
I20260809 04:44:27.016347 29076 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:27.016450 29076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:27.016685 29076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:27.018862 29076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "adfc94542c4c4ffdb2f371c8b939b9b5"
format_stamp: "Formatted at 2026-08-09 04:44:27 on dist-test-slave-vxj2"
server_key: "b7f6839b3186784c7161db462d80cd5b"
server_key_iv: "6ebf630cbd341e52b367bf34336b662d"
server_key_version: "encryptionkey@0"
I20260809 04:44:27.019462 29076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "adfc94542c4c4ffdb2f371c8b939b9b5"
format_stamp: "Formatted at 2026-08-09 04:44:27 on dist-test-slave-vxj2"
server_key: "b7f6839b3186784c7161db462d80cd5b"
server_key_iv: "6ebf630cbd341e52b367bf34336b662d"
server_key_version: "encryptionkey@0"
I20260809 04:44:27.023386 29076 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:27.026036 29092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:27.027345 29076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:44:27.027503 29076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "adfc94542c4c4ffdb2f371c8b939b9b5"
format_stamp: "Formatted at 2026-08-09 04:44:27 on dist-test-slave-vxj2"
server_key: "b7f6839b3186784c7161db462d80cd5b"
server_key_iv: "6ebf630cbd341e52b367bf34336b662d"
server_key_version: "encryptionkey@0"
I20260809 04:44:27.027622 29076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:27.041966 29076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:27.045323 29076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:27.045537 29076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:27.052958 29076 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37171
I20260809 04:44:27.053018 29147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37171 every 8 connection(s)
I20260809 04:44:27.053990 29076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:27.056972 29148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:27.063133 29148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5: Bootstrap starting.
I20260809 04:44:27.063239 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 29076
I20260809 04:44:27.063390 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:27.063725 6011 external_mini_cluster.cc:1428] Setting key 9ddca9b11bac52665b4bf16c07aae771
I20260809 04:44:27.066210 29148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:27.067054 29148 log.cc:824] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:27.069384 29148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5: No bootstrap required, opened a new log
I20260809 04:44:27.073523 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:27.065298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37953 (local address 127.0.0.1:37171)
0809 04:44:27.065774 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:27.065789 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:27.065826 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:44:27.066594 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:27.066613 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:27.066959 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:27.067309 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:27.067745 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:27.068785 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:27.070169 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:27.070657 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:27.070777 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:27.071696 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:27.071701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:27.071715 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:27.072099 (+ 384us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:27.072152 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:27.072174 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:27.072253 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:27.072476 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 04:44:27.072698 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":147,"threads_started":1}
I20260809 04:44:27.074049 29148 raft_consensus.cc:348] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfc94542c4c4ffdb2f371c8b939b9b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37171 } }
I20260809 04:44:27.074314 29148 raft_consensus.cc:374] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:27.074404 29148 raft_consensus.cc:729] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfc94542c4c4ffdb2f371c8b939b9b5, State: Initialized, Role: FOLLOWER
I20260809 04:44:27.074882 29148 consensus_queue.cc:260] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfc94542c4c4ffdb2f371c8b939b9b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37171 } }
I20260809 04:44:27.075052 29148 raft_consensus.cc:388] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:27.075152 29148 raft_consensus.cc:482] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:27.075295 29148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:27.076586 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:27.076608 29148 raft_consensus.cc:504] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfc94542c4c4ffdb2f371c8b939b9b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37171 } }
I20260809 04:44:27.076978 29148 leader_election.cc:302] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adfc94542c4c4ffdb2f371c8b939b9b5; no voters:
I20260809 04:44:27.077375 29148 leader_election.cc:288] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:27.077476 29153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:27.077733 29153 raft_consensus.cc:686] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [term 1 LEADER]: Becoming Leader. State: Replica: adfc94542c4c4ffdb2f371c8b939b9b5, State: Running, Role: LEADER
I20260809 04:44:27.078109 29153 consensus_queue.cc:237] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfc94542c4c4ffdb2f371c8b939b9b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37171 } }
I20260809 04:44:27.078852 29148 sys_catalog.cc:564] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:27.079857 29154 sys_catalog.cc:455] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adfc94542c4c4ffdb2f371c8b939b9b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfc94542c4c4ffdb2f371c8b939b9b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37171 } } }
I20260809 04:44:27.080044 29154 sys_catalog.cc:458] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:27.080456 29155 sys_catalog.cc:455] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adfc94542c4c4ffdb2f371c8b939b9b5. Latest consensus state: current_term: 1 leader_uuid: "adfc94542c4c4ffdb2f371c8b939b9b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfc94542c4c4ffdb2f371c8b939b9b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37171 } } }
I20260809 04:44:27.080610 29155 sys_catalog.cc:458] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:27.081341 29168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:27.084683 29168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:27.085188 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:27.079610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33593 (local address 127.0.0.1:37171)
0809 04:44:27.079705 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:27.079709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:27.080466 (+ 757us) server_negotiation.cc:407] Connection header received
0809 04:44:27.080651 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:27.080654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:27.080703 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:27.080758 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:27.081239 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:27.082004 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:27.083173 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:27.083539 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:27.083667 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:27.084593 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:27.084598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:27.084601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:27.084615 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:27.084648 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:27.084664 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:27.084779 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:27.084944 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 04:44:27.085005 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 04:44:27.090490 29168 catalog_manager.cc:1347] Generated new cluster ID: 93bd9fe76d25441a81c2b72b0fd68725
I20260809 04:44:27.090564 29168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:27.107822 29168 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:27.108795 29168 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:27.115593 29168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adfc94542c4c4ffdb2f371c8b939b9b5: Generated new TSK 0
I20260809 04:44:27.116286 29168 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:27.124722 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 29076
2026-08-09T04:44:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:27 dist-test-slave-vxj2 krb5kdc[29180](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:27 dist-test-slave-vxj2 krb5kdc[29180](info): set up 2 sockets
Aug 09 04:44:27 dist-test-slave-vxj2 krb5kdc[29180](info): commencing operation
krb5kdc: starting...
W20260809 04:44:28.223392 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:28 dist-test-slave-vxj2 krb5kdc[29180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:44:28.347879 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:28.464915 29196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:28.465250 29196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:28.465368 29196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:28.469403 29196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:28.469496 29196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:28.469534 29196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:28.469571 29196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:28.469602 29196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:28.474896 29196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:28.475999 29196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:28.477106 29196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:28.485908 29204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:28.485924 29201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:28.486012 29202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:28.486339 29196 server_base.cc:1034] running on GCE node
I20260809 04:44:28.486907 29196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:28.487939 29196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:28.489141 29196 hybrid_clock.cc:648] HybridClock initialized: now 1786250668489131 us; error 29 us; skew 500 ppm
Aug 09 04:44:28 dist-test-slave-vxj2 krb5kdc[29180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250668, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:28.492025 29196 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:44:28.492888 29196 webserver.cc:460] Webserver started at http://127.0.0.1:38533/ using document root <none> and password file <none>
I20260809 04:44:28.493683 29196 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:28.493796 29196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:28.494046 29196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:28.496311 29196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "ecd8b82ba7c14e1f8e7fa8eaab1e34a9"
format_stamp: "Formatted at 2026-08-09 04:44:28 on dist-test-slave-vxj2"
server_key: "93c453689e209cecc910467097369611"
server_key_iv: "c28ecc9ee51e40c520bb26f2b6fe3e00"
server_key_version: "encryptionkey@0"
I20260809 04:44:28.496933 29196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "ecd8b82ba7c14e1f8e7fa8eaab1e34a9"
format_stamp: "Formatted at 2026-08-09 04:44:28 on dist-test-slave-vxj2"
server_key: "93c453689e209cecc910467097369611"
server_key_iv: "c28ecc9ee51e40c520bb26f2b6fe3e00"
server_key_version: "encryptionkey@0"
I20260809 04:44:28.500792 29196 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 04:44:28.503417 29211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:28.504647 29196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:44:28.504801 29196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "ecd8b82ba7c14e1f8e7fa8eaab1e34a9"
format_stamp: "Formatted at 2026-08-09 04:44:28 on dist-test-slave-vxj2"
server_key: "93c453689e209cecc910467097369611"
server_key_iv: "c28ecc9ee51e40c520bb26f2b6fe3e00"
server_key_version: "encryptionkey@0"
I20260809 04:44:28.504923 29196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:28.509934 29196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:28.513062 29196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:28.513345 29196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:28.521083 29196 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39715
I20260809 04:44:28.521093 29266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39715 every 8 connection(s)
I20260809 04:44:28.522130 29196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:28.524881 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 29196
I20260809 04:44:28.525041 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:28.525285 29267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:28.525445 6011 external_mini_cluster.cc:1428] Setting key b9ee7942b40ab6c6e33a6c5abd1cbc3b
I20260809 04:44:28.531183 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:28.526882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46877 (local address 127.0.0.1:39715)
0809 04:44:28.527333 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:28.527354 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:28.527594 (+ 240us) server_negotiation.cc:407] Connection header received
0809 04:44:28.528465 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:28.528482 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:28.528770 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:28.529129 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:28.529969 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:28.529974 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:28.529986 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:28.529996 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:28.530046 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:28.530065 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:28.530232 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:28.530319 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 04:44:28.530483 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":140,"threads_started":1}
I20260809 04:44:28.532191 29267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9: Bootstrap starting.
I20260809 04:44:28.534108 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:28.534708 29267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:28.535418 29267 log.cc:824] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:28.537582 29267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9: No bootstrap required, opened a new log
I20260809 04:44:28.538897 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:28.536536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45037 (local address 127.0.0.1:39715)
0809 04:44:28.536657 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:28.536661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:28.537476 (+ 815us) server_negotiation.cc:407] Connection header received
0809 04:44:28.537592 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:28.537595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:28.537625 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:28.537727 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:28.538399 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:28.538402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:28.538404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:28.538414 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:28.538439 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:28.538449 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:28.538525 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:28.538677 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 04:44:28.538745 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:44:28.541683 29267 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd8b82ba7c14e1f8e7fa8eaab1e34a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39715 } }
I20260809 04:44:28.541945 29267 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:28.542006 29267 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd8b82ba7c14e1f8e7fa8eaab1e34a9, State: Initialized, Role: FOLLOWER
I20260809 04:44:28.542441 29267 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd8b82ba7c14e1f8e7fa8eaab1e34a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39715 } }
I20260809 04:44:28.542590 29267 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:28.542655 29267 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:28.542749 29267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:28.543668 29267 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd8b82ba7c14e1f8e7fa8eaab1e34a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39715 } }
I20260809 04:44:28.544013 29267 leader_election.cc:302] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecd8b82ba7c14e1f8e7fa8eaab1e34a9; no voters:
I20260809 04:44:28.544343 29267 leader_election.cc:288] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:28.544416 29278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:28.544621 29278 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [term 1 LEADER]: Becoming Leader. State: Replica: ecd8b82ba7c14e1f8e7fa8eaab1e34a9, State: Running, Role: LEADER
I20260809 04:44:28.544957 29278 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd8b82ba7c14e1f8e7fa8eaab1e34a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39715 } }
I20260809 04:44:28.545658 29267 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:28.546664 29279 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecd8b82ba7c14e1f8e7fa8eaab1e34a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd8b82ba7c14e1f8e7fa8eaab1e34a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39715 } } }
I20260809 04:44:28.546813 29279 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:28.547168 29280 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecd8b82ba7c14e1f8e7fa8eaab1e34a9. Latest consensus state: current_term: 1 leader_uuid: "ecd8b82ba7c14e1f8e7fa8eaab1e34a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd8b82ba7c14e1f8e7fa8eaab1e34a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39715 } } }
I20260809 04:44:28.547238 29287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:28.547258 29280 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:28.550467 29287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:28.555779 29287 catalog_manager.cc:1347] Generated new cluster ID: da9d3335702e4ce7882bdb0781bea3b4
I20260809 04:44:28.555863 29287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:28.571453 29287 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:28.571803 29287 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:28.581327 29287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecd8b82ba7c14e1f8e7fa8eaab1e34a9: Generated new TSK 0
I20260809 04:44:28.582111 29287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:28.587283 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 29196
2026-08-09T04:44:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1463 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:28 dist-test-slave-vxj2 krb5kdc[29300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:28 dist-test-slave-vxj2 krb5kdc[29300](info): set up 2 sockets
Aug 09 04:44:28 dist-test-slave-vxj2 krb5kdc[29300](info): commencing operation
krb5kdc: starting...
W20260809 04:44:29.690981 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:29 dist-test-slave-vxj2 krb5kdc[29300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:44:29.816593 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43837
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:29.926129 29313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:29.926420 29313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:29.926553 29313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:29.930626 29313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:29.930713 29313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:29.930750 29313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:29.930809 29313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:29.930861 29313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:29.936255 29313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:29.937397 29313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:29.938449 29313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:29.946790 29319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:29.946791 29321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:29.946854 29318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:29.947245 29313 server_base.cc:1034] running on GCE node
I20260809 04:44:29.947780 29313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:29.948838 29313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:29.950023 29313 hybrid_clock.cc:648] HybridClock initialized: now 1786250669950010 us; error 37 us; skew 500 ppm
Aug 09 04:44:29 dist-test-slave-vxj2 krb5kdc[29300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250669, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:29.959350 29313 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:44:29.960455 29313 webserver.cc:460] Webserver started at http://127.0.0.1:40749/ using document root <none> and password file <none>
I20260809 04:44:29.961041 29313 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:29.961143 29313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:29.961463 29313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:29.963577 29313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "06e75e59014b474895603b52fa91d719"
format_stamp: "Formatted at 2026-08-09 04:44:29 on dist-test-slave-vxj2"
server_key: "5ffe25051502ce321fceb096e009fdd7"
server_key_iv: "4365e4acef7e76e4ff74ea9792443b61"
server_key_version: "encryptionkey@0"
I20260809 04:44:29.964177 29313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "06e75e59014b474895603b52fa91d719"
format_stamp: "Formatted at 2026-08-09 04:44:29 on dist-test-slave-vxj2"
server_key: "5ffe25051502ce321fceb096e009fdd7"
server_key_iv: "4365e4acef7e76e4ff74ea9792443b61"
server_key_version: "encryptionkey@0"
I20260809 04:44:29.968092 29313 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:44:29.970532 29328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:29.971768 29313 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:44:29.971913 29313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "06e75e59014b474895603b52fa91d719"
format_stamp: "Formatted at 2026-08-09 04:44:29 on dist-test-slave-vxj2"
server_key: "5ffe25051502ce321fceb096e009fdd7"
server_key_iv: "4365e4acef7e76e4ff74ea9792443b61"
server_key_version: "encryptionkey@0"
I20260809 04:44:29.972033 29313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:29.995091 29313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:29.998522 29313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:29.998749 29313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:30.006228 29313 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43837
I20260809 04:44:30.006311 29384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43837 every 8 connection(s)
I20260809 04:44:30.007339 29313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:30.010363 29385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:30.013082 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 29313
I20260809 04:44:30.013273 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:30.013590 6011 external_mini_cluster.cc:1428] Setting key 75d40f2f3f28e41835e49abcca23d7fd
I20260809 04:44:30.017390 29385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719: Bootstrap starting.
I20260809 04:44:30.020077 29385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:30.020952 29385 log.cc:824] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:30.023180 29385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719: No bootstrap required, opened a new log
Aug 09 04:44:30 dist-test-slave-vxj2 krb5kdc[29300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 04:44:30.027513 29385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e75e59014b474895603b52fa91d719" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43837 } }
I20260809 04:44:30.027818 29385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:30.027913 29385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06e75e59014b474895603b52fa91d719, State: Initialized, Role: FOLLOWER
I20260809 04:44:30.028462 29385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e75e59014b474895603b52fa91d719" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43837 } }
I20260809 04:44:30.028621 29385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:30.028733 29385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:30.028853 29385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:30.030086 29385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e75e59014b474895603b52fa91d719" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43837 } }
I20260809 04:44:30.030436 29385 leader_election.cc:302] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06e75e59014b474895603b52fa91d719; no voters:
I20260809 04:44:30.030720 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.015042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:34581 (local address 127.0.0.1:43837)
0809 04:44:30.015537 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:30.015557 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:30.015602 (+ 45us) server_negotiation.cc:407] Connection header received
0809 04:44:30.016452 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:30.016477 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:30.016905 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:30.017296 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:30.018092 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.019605 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.020573 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.021297 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.021479 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.024309 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:30.024316 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:30.024340 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:30.024372 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:30.027103 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:30.027645 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:30.027651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:30.027657 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:30.027797 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:30.028272 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:30.028276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:30.028277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:30.028831 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:30.029047 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:30.029408 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 04:44:30.029630 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":163,"threads_started":1}
I20260809 04:44:30.030792 29385 leader_election.cc:288] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:30.031006 29391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:30.031231 29391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [term 1 LEADER]: Becoming Leader. State: Replica: 06e75e59014b474895603b52fa91d719, State: Running, Role: LEADER
I20260809 04:44:30.031555 29391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e75e59014b474895603b52fa91d719" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43837 } }
I20260809 04:44:30.032151 29385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:30.033286 29392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06e75e59014b474895603b52fa91d719" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e75e59014b474895603b52fa91d719" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43837 } } }
I20260809 04:44:30.033413 29392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:30.033757 29393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06e75e59014b474895603b52fa91d719. Latest consensus state: current_term: 1 leader_uuid: "06e75e59014b474895603b52fa91d719" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e75e59014b474895603b52fa91d719" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43837 } } }
I20260809 04:44:30.033885 29393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:30.034204 29400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:30.034301 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:30.037493 29400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:30.043936 29400 catalog_manager.cc:1347] Generated new cluster ID: 132dc7bf86e94eb6ac13a3222e7be13a
I20260809 04:44:30.043998 29400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:30.045980 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.036559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:53601 (local address 127.0.0.1:43837)
0809 04:44:30.036659 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:30.036663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:30.037681 (+ 1018us) server_negotiation.cc:407] Connection header received
0809 04:44:30.037810 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:30.037813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:30.037861 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:30.037953 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:30.038822 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.039547 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.040359 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.040605 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.040717 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.042360 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:30.042364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:30.042369 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:30.042394 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:30.044112 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:30.044603 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:30.044606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:30.044608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:30.044694 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:30.045033 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:30.045036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:30.045038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:30.045289 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:30.045462 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:30.045720 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 04:44:30.045812 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 04:44:30.079704 29400 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:30.080708 29400 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:30.084687 29400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06e75e59014b474895603b52fa91d719: Generated new TSK 0
I20260809 04:44:30.085441 29400 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:30.090965 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 29313
W20260809 04:44:30.099655 29298 connection.cc:535] client connection to 127.0.0.1:43837 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43837): Connection reset by peer (error 104)
2026-08-09T04:44:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:30 dist-test-slave-vxj2 krb5kdc[29418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:30 dist-test-slave-vxj2 krb5kdc[29418](info): set up 2 sockets
Aug 09 04:44:30 dist-test-slave-vxj2 krb5kdc[29418](info): commencing operation
krb5kdc: starting...
W20260809 04:44:31.236706 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:31 dist-test-slave-vxj2 krb5kdc[29418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:44:31.367930 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38141
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:31.487401 29432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:31.487649 29432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:31.487691 29432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:31.491716 29432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:31.491779 29432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:31.491799 29432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:31.491816 29432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:31.491832 29432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:31.497025 29432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:31.498070 29432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:31.499068 29432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:31.507602 29438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:31.507634 29437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:31.507716 29432 server_base.cc:1034] running on GCE node
W20260809 04:44:31.507612 29440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:31.508498 29432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:31.509663 29432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:31.510857 29432 hybrid_clock.cc:648] HybridClock initialized: now 1786250671510833 us; error 36 us; skew 500 ppm
Aug 09 04:44:31 dist-test-slave-vxj2 krb5kdc[29418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250671, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:31.513748 29432 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:44:31.514659 29432 webserver.cc:460] Webserver started at http://127.0.0.1:40561/ using document root <none> and password file <none>
I20260809 04:44:31.515211 29432 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:31.515302 29432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:31.515534 29432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:31.517606 29432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "bba818397e7c4abd9d200af7ebc81b67"
format_stamp: "Formatted at 2026-08-09 04:44:31 on dist-test-slave-vxj2"
server_key: "f1ca7e2adfaf15905d6754cf10b4e78c"
server_key_iv: "4c146b8b83dec1b18625a90dae9a792a"
server_key_version: "encryptionkey@0"
I20260809 04:44:31.518172 29432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "bba818397e7c4abd9d200af7ebc81b67"
format_stamp: "Formatted at 2026-08-09 04:44:31 on dist-test-slave-vxj2"
server_key: "f1ca7e2adfaf15905d6754cf10b4e78c"
server_key_iv: "4c146b8b83dec1b18625a90dae9a792a"
server_key_version: "encryptionkey@0"
I20260809 04:44:31.521814 29432 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:31.524191 29447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:31.525465 29432 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:31.525602 29432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "bba818397e7c4abd9d200af7ebc81b67"
format_stamp: "Formatted at 2026-08-09 04:44:31 on dist-test-slave-vxj2"
server_key: "f1ca7e2adfaf15905d6754cf10b4e78c"
server_key_iv: "4c146b8b83dec1b18625a90dae9a792a"
server_key_version: "encryptionkey@0"
I20260809 04:44:31.525718 29432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:31.536561 29432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:31.539819 29432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:31.540040 29432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:31.547262 29432 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38141
I20260809 04:44:31.547326 29500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38141 every 8 connection(s)
I20260809 04:44:31.548275 29432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:31.551153 29501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:31.554607 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 29432
I20260809 04:44:31.554769 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:31.555047 6011 external_mini_cluster.cc:1428] Setting key dbe05400f5853fba774d7ee53a9ecda6
I20260809 04:44:31.557132 29501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67: Bootstrap starting.
I20260809 04:44:31.559571 29501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:31.560328 29501 log.cc:824] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:31.562390 29501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67: No bootstrap required, opened a new log
I20260809 04:44:31.563995 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.556395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:60453 (local address 127.0.0.1:38141)
0809 04:44:31.556825 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.556841 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.556878 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:44:31.557603 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.557622 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.557965 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.558236 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:31.558634 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.559614 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.560616 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.561097 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.561368 (+ 271us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.562306 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:31.562311 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:31.562324 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:31.562751 (+ 427us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:31.562806 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:31.562828 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:31.562983 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.563168 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.563307 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":126,"threads_started":1}
I20260809 04:44:31.566506 29501 raft_consensus.cc:348] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bba818397e7c4abd9d200af7ebc81b67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38141 } }
I20260809 04:44:31.566776 29501 raft_consensus.cc:374] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:31.566860 29501 raft_consensus.cc:729] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bba818397e7c4abd9d200af7ebc81b67, State: Initialized, Role: FOLLOWER
I20260809 04:44:31.567049 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:31.567314 29501 consensus_queue.cc:260] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bba818397e7c4abd9d200af7ebc81b67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38141 } }
I20260809 04:44:31.567438 29501 raft_consensus.cc:388] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:31.567523 29501 raft_consensus.cc:482] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:31.567646 29501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:31.568619 29501 raft_consensus.cc:504] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bba818397e7c4abd9d200af7ebc81b67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38141 } }
I20260809 04:44:31.568953 29501 leader_election.cc:302] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bba818397e7c4abd9d200af7ebc81b67; no voters:
I20260809 04:44:31.569283 29501 leader_election.cc:288] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:31.569350 29512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:31.569612 29512 raft_consensus.cc:686] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [term 1 LEADER]: Becoming Leader. State: Replica: bba818397e7c4abd9d200af7ebc81b67, State: Running, Role: LEADER
I20260809 04:44:31.569977 29512 consensus_queue.cc:237] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bba818397e7c4abd9d200af7ebc81b67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38141 } }
I20260809 04:44:31.570636 29501 sys_catalog.cc:564] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:31.571694 29515 sys_catalog.cc:455] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bba818397e7c4abd9d200af7ebc81b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bba818397e7c4abd9d200af7ebc81b67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38141 } } }
I20260809 04:44:31.571823 29515 sys_catalog.cc:458] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:31.572217 29523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:31.572561 29514 sys_catalog.cc:455] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bba818397e7c4abd9d200af7ebc81b67. Latest consensus state: current_term: 1 leader_uuid: "bba818397e7c4abd9d200af7ebc81b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bba818397e7c4abd9d200af7ebc81b67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38141 } } }
I20260809 04:44:31.572728 29514 sys_catalog.cc:458] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:31.575001 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.569299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:42555 (local address 127.0.0.1:38141)
0809 04:44:31.569416 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.569420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.569798 (+ 378us) server_negotiation.cc:407] Connection header received
0809 04:44:31.569834 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.569837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.569889 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.569958 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:31.570359 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.571057 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.572876 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.573234 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.573339 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.574432 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:31.574436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:31.574439 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:31.574451 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:31.574478 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:31.574490 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:31.574569 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.574802 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.574859 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:44:31.576050 29523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:31.581211 29523 catalog_manager.cc:1347] Generated new cluster ID: 32a3733f6d4b410bb0b194e41329dc43
I20260809 04:44:31.581274 29523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:31.590852 29523 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:31.591747 29523 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:31.598289 29523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bba818397e7c4abd9d200af7ebc81b67: Generated new TSK 0
I20260809 04:44:31.599054 29523 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:31.602473 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 29432
W20260809 04:44:31.610666 29416 connection.cc:535] client connection to 127.0.0.1:38141 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38141): Connection reset by peer (error 104)
2026-08-09T04:44:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:31 dist-test-slave-vxj2 krb5kdc[29537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:31 dist-test-slave-vxj2 krb5kdc[29537](info): set up 2 sockets
Aug 09 04:44:31 dist-test-slave-vxj2 krb5kdc[29537](info): commencing operation
krb5kdc: starting...
W20260809 04:44:32.737668 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:32 dist-test-slave-vxj2 krb5kdc[29537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:44:32.862285 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36351
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:32.976496 29553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:32.976774 29553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:32.976854 29553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:32.980952 29553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:32.981020 29553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:32.981043 29553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:32.981094 29553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:32.981134 29553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:32.986968 29553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:32.988092 29553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:32.989113 29553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:32.997676 29561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:32.997679 29559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:32.997793 29553 server_base.cc:1034] running on GCE node
W20260809 04:44:32.997843 29558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:32.998658 29553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:32.999730 29553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:33.000921 29553 hybrid_clock.cc:648] HybridClock initialized: now 1786250673000900 us; error 35 us; skew 500 ppm
Aug 09 04:44:32 dist-test-slave-vxj2 krb5kdc[29537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250672, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:33.003713 29553 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:44:33.004551 29553 webserver.cc:460] Webserver started at http://127.0.0.1:42513/ using document root <none> and password file <none>
I20260809 04:44:33.005069 29553 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:33.005127 29553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:33.005373 29553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:33.007443 29553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "10577e32da2840efab8d1209907289a4"
format_stamp: "Formatted at 2026-08-09 04:44:33 on dist-test-slave-vxj2"
server_key: "a8e0bef3f0c6534cf66b6873462d54c6"
server_key_iv: "d7e627639afb7d8cac914329720ed172"
server_key_version: "encryptionkey@0"
I20260809 04:44:33.007965 29553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "10577e32da2840efab8d1209907289a4"
format_stamp: "Formatted at 2026-08-09 04:44:33 on dist-test-slave-vxj2"
server_key: "a8e0bef3f0c6534cf66b6873462d54c6"
server_key_iv: "d7e627639afb7d8cac914329720ed172"
server_key_version: "encryptionkey@0"
I20260809 04:44:33.011891 29553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:44:33.014295 29569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:33.015527 29553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:44:33.015673 29553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "10577e32da2840efab8d1209907289a4"
format_stamp: "Formatted at 2026-08-09 04:44:33 on dist-test-slave-vxj2"
server_key: "a8e0bef3f0c6534cf66b6873462d54c6"
server_key_iv: "d7e627639afb7d8cac914329720ed172"
server_key_version: "encryptionkey@0"
I20260809 04:44:33.015800 29553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:33.020442 29553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:33.023337 29553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:33.023519 29553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:33.030617 29553 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36351
I20260809 04:44:33.030642 29627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36351 every 8 connection(s)
I20260809 04:44:33.031535 29553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:33.034669 29628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:33.038523 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 29553
I20260809 04:44:33.038681 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:33.039002 6011 external_mini_cluster.cc:1428] Setting key 82ca94d9daec7966dc4142596c077eec
I20260809 04:44:33.041576 29628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4: Bootstrap starting.
I20260809 04:44:33.044119 29628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:33.044404 29631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:33.040525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:47749 (local address 127.0.0.1:36351)
0809 04:44:33.040984 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:33.040999 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:33.041034 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:44:33.041803 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:33.041821 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:33.042091 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:33.042427 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:33.043216 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:33.043221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:33.043231 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:33.043243 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:33.043296 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:33.043315 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:33.043452 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:33.043525 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 04:44:33.043684 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":140,"threads_started":1}
I20260809 04:44:33.045089 29628 log.cc:824] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:33.047355 29628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4: No bootstrap required, opened a new log
I20260809 04:44:33.047634 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:33.051508 29631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:33.049762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:58387 (local address 127.0.0.1:36351)
0809 04:44:33.049883 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:33.049886 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:33.050130 (+ 244us) server_negotiation.cc:407] Connection header received
0809 04:44:33.050231 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:33.050234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:33.050265 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:33.050334 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:33.051113 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:33.051116 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:33.051118 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:33.051128 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:33.051152 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:33.051162 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:33.051232 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:33.051335 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 04:44:33.051367 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:44:33.051900 29628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10577e32da2840efab8d1209907289a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36351 } }
I20260809 04:44:33.052142 29628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:33.052207 29628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10577e32da2840efab8d1209907289a4, State: Initialized, Role: FOLLOWER
I20260809 04:44:33.052663 29628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10577e32da2840efab8d1209907289a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36351 } }
I20260809 04:44:33.052824 29628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:33.052896 29628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:33.053018 29628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:33.053978 29628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10577e32da2840efab8d1209907289a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36351 } }
I20260809 04:44:33.054319 29628 leader_election.cc:302] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10577e32da2840efab8d1209907289a4; no voters:
I20260809 04:44:33.054632 29628 leader_election.cc:288] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:33.054709 29638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:33.054994 29638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [term 1 LEADER]: Becoming Leader. State: Replica: 10577e32da2840efab8d1209907289a4, State: Running, Role: LEADER
I20260809 04:44:33.055351 29638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10577e32da2840efab8d1209907289a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36351 } }
I20260809 04:44:33.055894 29628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:33.057183 29639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10577e32da2840efab8d1209907289a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10577e32da2840efab8d1209907289a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36351 } } }
I20260809 04:44:33.057225 29640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10577e32da2840efab8d1209907289a4. Latest consensus state: current_term: 1 leader_uuid: "10577e32da2840efab8d1209907289a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10577e32da2840efab8d1209907289a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36351 } } }
I20260809 04:44:33.057381 29639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:33.057386 29640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4 [sys.catalog]: This master's current role is: LEADER
W20260809 04:44:33.060787 29654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:33.060856 29654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:33.060935 29649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:33.061707 29649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:33.067308 29649 catalog_manager.cc:1347] Generated new cluster ID: 0467af10fcb64c078d7551bf111b4847
I20260809 04:44:33.067386 29649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:33.083302 29649 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:33.083642 29649 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:33.096167 29649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10577e32da2840efab8d1209907289a4: Generated new TSK 0
I20260809 04:44:33.096920 29649 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:33.108165 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 29553
2026-08-09T04:44:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:33 dist-test-slave-vxj2 krb5kdc[29659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:33 dist-test-slave-vxj2 krb5kdc[29659](info): set up 2 sockets
Aug 09 04:44:33 dist-test-slave-vxj2 krb5kdc[29659](info): commencing operation
krb5kdc: starting...
W20260809 04:44:34.290364 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:34 dist-test-slave-vxj2 krb5kdc[29659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:44:34.416491 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:34.531394 29673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:34.531725 29673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:34.531817 29673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:34.536046 29673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:34.536113 29673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:34.536134 29673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:34.536150 29673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:34.536191 29673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:34.541630 29673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43233
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:34.542878 29673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:34.543933 29673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:34.552510 29682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:34.552618 29681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:34.552613 29684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:34.552875 29673 server_base.cc:1034] running on GCE node
I20260809 04:44:34.553489 29673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:34.554590 29673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:34.555752 29673 hybrid_clock.cc:648] HybridClock initialized: now 1786250674555739 us; error 25 us; skew 500 ppm
Aug 09 04:44:34 dist-test-slave-vxj2 krb5kdc[29659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250674, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:34.558533 29673 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:44:34.559394 29673 webserver.cc:460] Webserver started at http://127.0.0.1:35993/ using document root <none> and password file <none>
I20260809 04:44:34.559962 29673 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:34.560024 29673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:34.560248 29673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:34.562629 29673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "e44fa3155a964b2e99f494cd999aa40d"
format_stamp: "Formatted at 2026-08-09 04:44:34 on dist-test-slave-vxj2"
server_key: "4b9c84910a2c6ddfeb97dcfd2c1d2381"
server_key_iv: "f14f33729bf190dc27df930d896e7366"
server_key_version: "encryptionkey@0"
I20260809 04:44:34.563252 29673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "e44fa3155a964b2e99f494cd999aa40d"
format_stamp: "Formatted at 2026-08-09 04:44:34 on dist-test-slave-vxj2"
server_key: "4b9c84910a2c6ddfeb97dcfd2c1d2381"
server_key_iv: "f14f33729bf190dc27df930d896e7366"
server_key_version: "encryptionkey@0"
I20260809 04:44:34.566984 29673 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:44:34.569454 29691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:34.570876 29673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:34.571048 29673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "e44fa3155a964b2e99f494cd999aa40d"
format_stamp: "Formatted at 2026-08-09 04:44:34 on dist-test-slave-vxj2"
server_key: "4b9c84910a2c6ddfeb97dcfd2c1d2381"
server_key_iv: "f14f33729bf190dc27df930d896e7366"
server_key_version: "encryptionkey@0"
I20260809 04:44:34.571171 29673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:34.598060 29673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:34.601310 29673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:34.601531 29673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:34.609076 29673 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43233
I20260809 04:44:34.609107 29747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43233 every 8 connection(s)
I20260809 04:44:34.610085 29673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:34.612851 29748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:34.613229 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 29673
I20260809 04:44:34.613360 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:34.613660 6011 external_mini_cluster.cc:1428] Setting key 61b6aebb200647f5c1bdf6d7063709ab
I20260809 04:44:34.620196 29748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d: Bootstrap starting.
I20260809 04:44:34.622836 29748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d: Neither blocks nor log segments found. Creating new log.
Aug 09 04:44:34 dist-test-slave-vxj2 krb5kdc[29659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 04:44:34.623612 29748 log.cc:824] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:34.626046 29748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d: No bootstrap required, opened a new log
I20260809 04:44:34.629568 29751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:34.615101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:44491 (local address 127.0.0.1:43233)
0809 04:44:34.615541 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:34.615560 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:34.615603 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:44:34.616348 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:34.616371 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:34.616829 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:34.617225 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:34.617926 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:34.619454 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:34.620303 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:34.620963 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:34.621121 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:34.623971 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:34.623981 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:34.624001 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:34.624031 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:34.626089 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:34.626633 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:34.626639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:34.626645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:34.626802 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:34.627189 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:34.627192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:34.627194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:34.627686 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:34.627918 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:34.628115 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 04:44:34.628375 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":111,"threads_started":1}
I20260809 04:44:34.630807 29748 raft_consensus.cc:348] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e44fa3155a964b2e99f494cd999aa40d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43233 } }
I20260809 04:44:34.631070 29748 raft_consensus.cc:374] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:34.631124 29748 raft_consensus.cc:729] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44fa3155a964b2e99f494cd999aa40d, State: Initialized, Role: FOLLOWER
I20260809 04:44:34.631551 29748 consensus_queue.cc:260] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e44fa3155a964b2e99f494cd999aa40d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43233 } }
I20260809 04:44:34.631719 29748 raft_consensus.cc:388] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:34.631788 29748 raft_consensus.cc:482] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:34.631887 29748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:34.632653 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:34.633033 29748 raft_consensus.cc:504] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e44fa3155a964b2e99f494cd999aa40d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43233 } }
I20260809 04:44:34.633430 29748 leader_election.cc:302] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e44fa3155a964b2e99f494cd999aa40d; no voters:
I20260809 04:44:34.633766 29748 leader_election.cc:288] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:34.633903 29753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:34.634137 29753 raft_consensus.cc:686] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [term 1 LEADER]: Becoming Leader. State: Replica: e44fa3155a964b2e99f494cd999aa40d, State: Running, Role: LEADER
I20260809 04:44:34.634555 29753 consensus_queue.cc:237] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e44fa3155a964b2e99f494cd999aa40d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43233 } }
I20260809 04:44:34.635062 29748 sys_catalog.cc:564] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:34.636313 29754 sys_catalog.cc:455] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e44fa3155a964b2e99f494cd999aa40d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e44fa3155a964b2e99f494cd999aa40d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43233 } } }
I20260809 04:44:34.636458 29754 sys_catalog.cc:458] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:34.636889 29757 sys_catalog.cc:455] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e44fa3155a964b2e99f494cd999aa40d. Latest consensus state: current_term: 1 leader_uuid: "e44fa3155a964b2e99f494cd999aa40d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e44fa3155a964b2e99f494cd999aa40d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43233 } } }
I20260809 04:44:34.636983 29757 sys_catalog.cc:458] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d [sys.catalog]: This master's current role is: LEADER
W20260809 04:44:34.639961 29773 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:34.640049 29773 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:34.640108 29768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:34.640936 29768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:34.646677 29751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:34.636167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:41527 (local address 127.0.0.1:43233)
0809 04:44:34.636305 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:34.636310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:34.637749 (+ 1439us) server_negotiation.cc:407] Connection header received
0809 04:44:34.637997 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:34.638001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:34.638057 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:34.638157 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:34.639002 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:34.639810 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:34.640930 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:34.641317 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:34.641431 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:34.643157 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:34.643161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:34.643163 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:34.643184 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:34.644620 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:34.645090 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:34.645098 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:34.645100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:34.645227 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:34.645602 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:34.645606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:34.645608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:34.645844 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:34.646028 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:34.646373 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 04:44:34.646505 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:44:34.647301 29768 catalog_manager.cc:1347] Generated new cluster ID: 64bc79ea2c3343b2b7f0adc01cfe8c15
I20260809 04:44:34.647403 29768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:34.658735 29768 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:34.659909 29768 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:34.666401 29768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e44fa3155a964b2e99f494cd999aa40d: Generated new TSK 0
I20260809 04:44:34.667161 29768 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:34.676604 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 29673
W20260809 04:44:34.683955 29657 connection.cc:535] client connection to 127.0.0.1:43233 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43233): Connection reset by peer (error 104)
2026-08-09T04:44:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1566 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:34 dist-test-slave-vxj2 krb5kdc[29781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:34 dist-test-slave-vxj2 krb5kdc[29781](info): set up 2 sockets
Aug 09 04:44:34 dist-test-slave-vxj2 krb5kdc[29781](info): commencing operation
krb5kdc: starting...
W20260809 04:44:35.792725 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:35 dist-test-slave-vxj2 krb5kdc[29781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:44:35.920156 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46631
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:36.037027 29795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:36.037309 29795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:36.037351 29795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:36.041340 29795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:36.041406 29795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:36.041426 29795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:36.041446 29795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:36.041471 29795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:36.046811 29795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46631
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:36.047766 29795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:36.048693 29795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:36.056974 29801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:36.057015 29800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:36.057015 29803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:36.057124 29795 server_base.cc:1034] running on GCE node
I20260809 04:44:36.057785 29795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:36.058835 29795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:36.060061 29795 hybrid_clock.cc:648] HybridClock initialized: now 1786250676060046 us; error 26 us; skew 500 ppm
Aug 09 04:44:36 dist-test-slave-vxj2 krb5kdc[29781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:36.066627 29795 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:44:36.067664 29795 webserver.cc:460] Webserver started at http://127.0.0.1:34319/ using document root <none> and password file <none>
I20260809 04:44:36.068259 29795 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:36.068356 29795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:36.068591 29795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:36.070734 29795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "b5454eb9b67f4220b6fa5c053a060864"
format_stamp: "Formatted at 2026-08-09 04:44:36 on dist-test-slave-vxj2"
server_key: "3d502c9db650cf859b09e4b5ced6e4f3"
server_key_iv: "d36a477a6305c93cb8e0b5b45e18a7f9"
server_key_version: "encryptionkey@0"
I20260809 04:44:36.071350 29795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "b5454eb9b67f4220b6fa5c053a060864"
format_stamp: "Formatted at 2026-08-09 04:44:36 on dist-test-slave-vxj2"
server_key: "3d502c9db650cf859b09e4b5ced6e4f3"
server_key_iv: "d36a477a6305c93cb8e0b5b45e18a7f9"
server_key_version: "encryptionkey@0"
I20260809 04:44:36.075260 29795 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:36.077586 29810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:36.078794 29795 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:36.078943 29795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "b5454eb9b67f4220b6fa5c053a060864"
format_stamp: "Formatted at 2026-08-09 04:44:36 on dist-test-slave-vxj2"
server_key: "3d502c9db650cf859b09e4b5ced6e4f3"
server_key_iv: "d36a477a6305c93cb8e0b5b45e18a7f9"
server_key_version: "encryptionkey@0"
I20260809 04:44:36.079048 29795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:36.098605 29795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:36.101933 29795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:36.102142 29795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:36.109539 29795 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46631
I20260809 04:44:36.109563 29870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46631 every 8 connection(s)
I20260809 04:44:36.110486 29795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:36.113240 29871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:36.116776 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 29795
I20260809 04:44:36.116909 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:36.117272 6011 external_mini_cluster.cc:1428] Setting key 177a06b79c7ae5afb123ce9fe4fcced9
I20260809 04:44:36.119258 29871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864: Bootstrap starting.
I20260809 04:44:36.121454 29871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:36.122184 29871 log.cc:824] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:36.123971 29871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864: No bootstrap required, opened a new log
I20260809 04:44:36.126176 29874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:36.118736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:41099 (local address 127.0.0.1:46631)
0809 04:44:36.119151 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:36.119166 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:36.119200 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:44:36.119884 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:36.119902 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:36.120229 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:36.120573 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:36.120990 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:36.122073 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:36.123015 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:36.123480 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:36.123642 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:36.124560 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:36.124571 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:36.124583 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:36.124955 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:36.125006 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:36.125028 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:36.125189 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:36.125377 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 04:44:36.125511 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":121,"threads_started":1}
I20260809 04:44:36.127561 29871 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5454eb9b67f4220b6fa5c053a060864" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46631 } }
I20260809 04:44:36.127794 29871 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:36.127884 29871 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5454eb9b67f4220b6fa5c053a060864, State: Initialized, Role: FOLLOWER
I20260809 04:44:36.128322 29871 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5454eb9b67f4220b6fa5c053a060864" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46631 } }
I20260809 04:44:36.128489 29871 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:36.128566 29871 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:36.128682 29871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:36.128976 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:36.129803 29871 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5454eb9b67f4220b6fa5c053a060864" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46631 } }
I20260809 04:44:36.130208 29871 leader_election.cc:302] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5454eb9b67f4220b6fa5c053a060864; no voters:
I20260809 04:44:36.130566 29871 leader_election.cc:288] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:36.130654 29879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:36.130925 29879 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [term 1 LEADER]: Becoming Leader. State: Replica: b5454eb9b67f4220b6fa5c053a060864, State: Running, Role: LEADER
I20260809 04:44:36.131281 29879 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5454eb9b67f4220b6fa5c053a060864" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46631 } }
I20260809 04:44:36.132004 29871 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:36.133042 29882 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5454eb9b67f4220b6fa5c053a060864" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5454eb9b67f4220b6fa5c053a060864" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46631 } } }
I20260809 04:44:36.133184 29882 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:36.133410 29881 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5454eb9b67f4220b6fa5c053a060864. Latest consensus state: current_term: 1 leader_uuid: "b5454eb9b67f4220b6fa5c053a060864" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5454eb9b67f4220b6fa5c053a060864" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46631 } } }
I20260809 04:44:36.133746 29881 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:36.134665 29891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:36.136374 29874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:36.131622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:54719 (local address 127.0.0.1:46631)
0809 04:44:36.131715 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:36.131719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:36.131822 (+ 103us) server_negotiation.cc:407] Connection header received
0809 04:44:36.131925 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:36.131928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:36.131973 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:36.132048 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:36.132345 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:36.133223 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:36.134396 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:36.134672 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:36.134803 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:36.135815 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:36.135819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:36.135821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:36.135835 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:36.135864 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:36.135883 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:36.136004 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:36.136189 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 04:44:36.136227 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 04:44:36.138067 29891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:36.143527 29891 catalog_manager.cc:1347] Generated new cluster ID: 284b6c90df87417d810a00c951f76b7a
I20260809 04:44:36.143608 29891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:36.155468 29891 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:36.156595 29891 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:36.163749 29891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5454eb9b67f4220b6fa5c053a060864: Generated new TSK 0
I20260809 04:44:36.164613 29891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:36.173893 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 29795
W20260809 04:44:36.182395 29779 connection.cc:535] client connection to 127.0.0.1:46631 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46631): Connection reset by peer (error 104)
2026-08-09T04:44:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:36 dist-test-slave-vxj2 krb5kdc[29905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:36 dist-test-slave-vxj2 krb5kdc[29905](info): set up 2 sockets
Aug 09 04:44:36 dist-test-slave-vxj2 krb5kdc[29905](info): commencing operation
krb5kdc: starting...
W20260809 04:44:37.260892 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:37 dist-test-slave-vxj2 krb5kdc[29905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:44:37.384127 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45551
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:37.495500 29920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:37.495770 29920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:37.495811 29920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:37.499926 29920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:37.499998 29920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:37.500018 29920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:37.500036 29920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:37.500051 29920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:37.505432 29920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45551
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:37.506558 29920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:37.507617 29920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:37.516664 29928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:37.516656 29926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:37.516851 29925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:37.516959 29920 server_base.cc:1034] running on GCE node
I20260809 04:44:37.517721 29920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:37.518893 29920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:37.520054 29920 hybrid_clock.cc:648] HybridClock initialized: now 1786250677520035 us; error 31 us; skew 500 ppm
Aug 09 04:44:37 dist-test-slave-vxj2 krb5kdc[29905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250677, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:37.522840 29920 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:44:37.523695 29920 webserver.cc:460] Webserver started at http://127.0.0.1:35415/ using document root <none> and password file <none>
I20260809 04:44:37.524214 29920 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:37.524271 29920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:37.524437 29920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:37.526510 29920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "70a51ef1dc50455dad4edc56cffd556a"
format_stamp: "Formatted at 2026-08-09 04:44:37 on dist-test-slave-vxj2"
server_key: "0d9bce9e522358693a642c2a5de00aa0"
server_key_iv: "39ccc849a3a89bdf31cfed3dc5f97e09"
server_key_version: "encryptionkey@0"
I20260809 04:44:37.527038 29920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "70a51ef1dc50455dad4edc56cffd556a"
format_stamp: "Formatted at 2026-08-09 04:44:37 on dist-test-slave-vxj2"
server_key: "0d9bce9e522358693a642c2a5de00aa0"
server_key_iv: "39ccc849a3a89bdf31cfed3dc5f97e09"
server_key_version: "encryptionkey@0"
I20260809 04:44:37.530747 29920 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 04:44:37.533098 29936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:37.534379 29920 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:44:37.534523 29920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "70a51ef1dc50455dad4edc56cffd556a"
format_stamp: "Formatted at 2026-08-09 04:44:37 on dist-test-slave-vxj2"
server_key: "0d9bce9e522358693a642c2a5de00aa0"
server_key_iv: "39ccc849a3a89bdf31cfed3dc5f97e09"
server_key_version: "encryptionkey@0"
I20260809 04:44:37.534636 29920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:37.539327 29920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:37.542644 29920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:37.542841 29920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:37.550237 29920 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45551
I20260809 04:44:37.550271 29995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45551 every 8 connection(s)
I20260809 04:44:37.551162 29920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:37.553902 29996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:37.559345 29996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a: Bootstrap starting.
I20260809 04:44:37.561275 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 29920
I20260809 04:44:37.561419 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:37.561738 6011 external_mini_cluster.cc:1428] Setting key 27b1e4b478097243104e060077ca208a
I20260809 04:44:37.562124 29996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:37.562971 29996 log.cc:824] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:37.564873 29996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a: No bootstrap required, opened a new log
I20260809 04:44:37.566860 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:37.562920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:36715 (local address 127.0.0.1:45551)
0809 04:44:37.563420 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:37.563439 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:37.563483 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:44:37.564206 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:37.564224 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:37.564501 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:37.564834 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:37.565708 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:37.565713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:37.565724 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:37.565734 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:37.565786 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:37.565805 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:37.565917 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:37.566028 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 04:44:37.566187 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":125,"threads_started":1}
I20260809 04:44:37.569041 29996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a51ef1dc50455dad4edc56cffd556a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } }
I20260809 04:44:37.569337 29996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:37.569414 29996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a51ef1dc50455dad4edc56cffd556a, State: Initialized, Role: FOLLOWER
I20260809 04:44:37.569702 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:37.569906 29996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a51ef1dc50455dad4edc56cffd556a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } }
I20260809 04:44:37.570057 29996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:37.570142 29996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:37.570240 29996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:37.571224 29996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a51ef1dc50455dad4edc56cffd556a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } }
I20260809 04:44:37.571568 29996 leader_election.cc:302] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70a51ef1dc50455dad4edc56cffd556a; no voters:
I20260809 04:44:37.571904 29996 leader_election.cc:288] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:37.571996 30005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:37.572269 30005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [term 1 LEADER]: Becoming Leader. State: Replica: 70a51ef1dc50455dad4edc56cffd556a, State: Running, Role: LEADER
I20260809 04:44:37.572606 30005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a51ef1dc50455dad4edc56cffd556a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } }
I20260809 04:44:37.573429 29996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:37.573812 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:37.571856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:57629 (local address 127.0.0.1:45551)
0809 04:44:37.572128 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:37.572133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:37.572199 (+ 66us) server_negotiation.cc:407] Connection header received
0809 04:44:37.572364 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:37.572368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:37.572401 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:37.572448 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:37.573374 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:37.573378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:37.573380 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:37.573392 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:37.573419 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:37.573431 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:37.573514 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:37.573653 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 04:44:37.573693 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260809 04:44:37.573781 30005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70a51ef1dc50455dad4edc56cffd556a. Latest consensus state: current_term: 1 leader_uuid: "70a51ef1dc50455dad4edc56cffd556a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a51ef1dc50455dad4edc56cffd556a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } } }
I20260809 04:44:37.573899 30005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:37.574626 30008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70a51ef1dc50455dad4edc56cffd556a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a51ef1dc50455dad4edc56cffd556a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45551 } } }
I20260809 04:44:37.574777 30008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:37.575088 30010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:37.578162 30010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:37.583382 30010 catalog_manager.cc:1347] Generated new cluster ID: 1ac7e8e4ce274dbf8627b68a107a884b
I20260809 04:44:37.583473 30010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:37.593636 30010 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:37.593971 30010 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:37.606585 30010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70a51ef1dc50455dad4edc56cffd556a: Generated new TSK 0
I20260809 04:44:37.607331 30010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:37.612198 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 29920
2026-08-09T04:44:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1436 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:37 dist-test-slave-vxj2 krb5kdc[30028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:37 dist-test-slave-vxj2 krb5kdc[30028](info): set up 2 sockets
Aug 09 04:44:37 dist-test-slave-vxj2 krb5kdc[30028](info): commencing operation
krb5kdc: starting...
W20260809 04:44:38.700547 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:38 dist-test-slave-vxj2 krb5kdc[30028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:44:38.823880 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:39407
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:39407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:38.935480 30041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:38.935750 30041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:38.935791 30041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:38.939797 30041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:38.939858 30041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:38.939878 30041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:38.939894 30041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:38.939910 30041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:38.945236 30041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:39407
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:39407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:38.946339 30041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:38.947412 30041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:38.956018 30049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:38.955988 30047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:38.956007 30046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:38.956516 30041 server_base.cc:1034] running on GCE node
I20260809 04:44:38.957054 30041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:38.958132 30041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:38.959307 30041 hybrid_clock.cc:648] HybridClock initialized: now 1786250678959286 us; error 45 us; skew 500 ppm
Aug 09 04:44:38 dist-test-slave-vxj2 krb5kdc[30028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250678, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:38.962142 30041 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:44:38.962985 30041 webserver.cc:460] Webserver started at http://127.5.222.254:41399/ using document root <none> and password file <none>
I20260809 04:44:38.963524 30041 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:38.963609 30041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:38.963826 30041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:38.965865 30041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "726d0f474d0f47eb9301f1215748dd6b"
format_stamp: "Formatted at 2026-08-09 04:44:38 on dist-test-slave-vxj2"
server_key: "310c6a605a43314ba508d47599efece5"
server_key_iv: "1324f120d325e99a540bb948dae57c14"
server_key_version: "encryptionkey@0"
I20260809 04:44:38.966429 30041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "726d0f474d0f47eb9301f1215748dd6b"
format_stamp: "Formatted at 2026-08-09 04:44:38 on dist-test-slave-vxj2"
server_key: "310c6a605a43314ba508d47599efece5"
server_key_iv: "1324f120d325e99a540bb948dae57c14"
server_key_version: "encryptionkey@0"
I20260809 04:44:38.970100 30041 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:44:38.972518 30057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:38.973790 30041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:38.973930 30041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "726d0f474d0f47eb9301f1215748dd6b"
format_stamp: "Formatted at 2026-08-09 04:44:38 on dist-test-slave-vxj2"
server_key: "310c6a605a43314ba508d47599efece5"
server_key_iv: "1324f120d325e99a540bb948dae57c14"
server_key_version: "encryptionkey@0"
I20260809 04:44:38.974049 30041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:38.983861 30041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:38.987051 30041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:38.987248 30041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:38.994421 30041 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:39407
I20260809 04:44:38.994481 30112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:39407 every 8 connection(s)
I20260809 04:44:38.995380 30041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:38.998118 30113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:39.000120 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 30041
I20260809 04:44:39.000275 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:39.000552 6011 external_mini_cluster.cc:1428] Setting key 1b26404a70691b618f22fe5fb3c5c6cf
I20260809 04:44:39.004590 30113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b: Bootstrap starting.
I20260809 04:44:39.007277 30113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:39.008165 30113 log.cc:824] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:39.010324 30113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b: No bootstrap required, opened a new log
Aug 09 04:44:39 dist-test-slave-vxj2 krb5kdc[30028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:39.013772 30113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726d0f474d0f47eb9301f1215748dd6b" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39407 } }
I20260809 04:44:39.014021 30113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:39.014099 30113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 726d0f474d0f47eb9301f1215748dd6b, State: Initialized, Role: FOLLOWER
I20260809 04:44:39.014598 30113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726d0f474d0f47eb9301f1215748dd6b" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39407 } }
I20260809 04:44:39.014750 30113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:39.014847 30113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:39.014947 30113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:39.015849 30113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726d0f474d0f47eb9301f1215748dd6b" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39407 } }
I20260809 04:44:39.016222 30113 leader_election.cc:302] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 726d0f474d0f47eb9301f1215748dd6b; no voters:
I20260809 04:44:39.016571 30113 leader_election.cc:288] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:39.016674 30119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:39.016903 30119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [term 1 LEADER]: Becoming Leader. State: Replica: 726d0f474d0f47eb9301f1215748dd6b, State: Running, Role: LEADER
I20260809 04:44:39.017045 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:39.002001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52813 (local address 127.5.222.254:39407)
0809 04:44:39.002426 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:39.002445 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:39.002488 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:44:39.003456 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:39.003480 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:39.003902 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:39.004260 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:39.004975 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:39.006465 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:39.007311 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:39.008027 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:39.008194 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:39.011191 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:39.011197 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:39.011210 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:39.011243 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:39.013668 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:39.014239 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:39.014250 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:39.014254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:39.014387 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:39.014724 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:39.014745 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:39.014750 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:39.015272 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:39.015504 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:39.015820 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 04:44:39.016054 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":98,"threads_started":1}
I20260809 04:44:39.017251 30119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726d0f474d0f47eb9301f1215748dd6b" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39407 } }
I20260809 04:44:39.017874 30113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:39.018385 30121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 726d0f474d0f47eb9301f1215748dd6b. Latest consensus state: current_term: 1 leader_uuid: "726d0f474d0f47eb9301f1215748dd6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726d0f474d0f47eb9301f1215748dd6b" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39407 } } }
I20260809 04:44:39.018496 30121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:39.020417 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:39.019740 30120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "726d0f474d0f47eb9301f1215748dd6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726d0f474d0f47eb9301f1215748dd6b" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39407 } } }
I20260809 04:44:39.021432 30120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b [sys.catalog]: This master's current role is: LEADER
W20260809 04:44:39.023006 30137 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:39.023094 30137 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:39.023176 30123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:39.024346 30123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:39.030836 30123 catalog_manager.cc:1347] Generated new cluster ID: a81e7ff5b5284124837a8dbeaf8c2ff1
I20260809 04:44:39.030905 30123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:39.033049 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:39.023579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60915 (local address 127.5.222.254:39407)
0809 04:44:39.023703 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:39.023707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:39.024051 (+ 344us) server_negotiation.cc:407] Connection header received
0809 04:44:39.024161 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:39.024165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:39.024210 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:39.024299 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:39.025191 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:39.026013 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:39.027066 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:39.027293 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:39.027402 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:39.028908 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:39.028913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:39.028916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:39.028942 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:39.030751 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:39.031459 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:39.031466 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:39.031467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:39.031556 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:39.031987 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:39.031991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:39.031993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:39.032203 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:39.032334 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:39.032782 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 04:44:39.032891 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:44:39.038393 30123 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:39.039525 30123 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:39.049469 30123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 726d0f474d0f47eb9301f1215748dd6b: Generated new TSK 0
I20260809 04:44:39.050268 30123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:39.061378 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 30041
W20260809 04:44:39.070545 30026 connection.cc:535] client connection to 127.5.222.254:39407 recv error: Network error: failed to read from TLS socket (remote: 127.5.222.254:39407): Connection reset by peer (error 104)
2026-08-09T04:44:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1452 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:39 dist-test-slave-vxj2 krb5kdc[30149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:39 dist-test-slave-vxj2 krb5kdc[30149](info): set up 2 sockets
Aug 09 04:44:39 dist-test-slave-vxj2 krb5kdc[30149](info): commencing operation
krb5kdc: starting...
W20260809 04:44:40.192319 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:40 dist-test-slave-vxj2 krb5kdc[30149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:44:40.315632 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:39553
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:35375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:39553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:40.428561 30163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:40.428930 30163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:40.429047 30163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:40.433261 30163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:40.433357 30163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:40.433395 30163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:40.433446 30163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:40.433493 30163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:40.438813 30163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:35375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:39553
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:39553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:40.439898 30163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:40.440929 30163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:40.449451 30169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:40.449451 30171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:40.449470 30168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:40.449707 30163 server_base.cc:1034] running on GCE node
I20260809 04:44:40.450374 30163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:40.451493 30163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:40.452688 30163 hybrid_clock.cc:648] HybridClock initialized: now 1786250680452667 us; error 34 us; skew 500 ppm
Aug 09 04:44:40 dist-test-slave-vxj2 krb5kdc[30149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250680, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:40.455725 30163 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:44:40.456559 30163 webserver.cc:460] Webserver started at http://127.5.222.254:45041/ using document root <none> and password file <none>
I20260809 04:44:40.457067 30163 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:40.457129 30163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:40.457370 30163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:40.459406 30163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "eba54494683b470b972ec2f2ae210166"
format_stamp: "Formatted at 2026-08-09 04:44:40 on dist-test-slave-vxj2"
server_key: "05f49490f675190a9d2a82802d3467c4"
server_key_iv: "af8ecace81cc00439a5ebfcc2fba09ab"
server_key_version: "encryptionkey@0"
I20260809 04:44:40.459950 30163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "eba54494683b470b972ec2f2ae210166"
format_stamp: "Formatted at 2026-08-09 04:44:40 on dist-test-slave-vxj2"
server_key: "05f49490f675190a9d2a82802d3467c4"
server_key_iv: "af8ecace81cc00439a5ebfcc2fba09ab"
server_key_version: "encryptionkey@0"
I20260809 04:44:40.463697 30163 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:44:40.466159 30178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:40.467418 30163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:40.467526 30163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "eba54494683b470b972ec2f2ae210166"
format_stamp: "Formatted at 2026-08-09 04:44:40 on dist-test-slave-vxj2"
server_key: "05f49490f675190a9d2a82802d3467c4"
server_key_iv: "af8ecace81cc00439a5ebfcc2fba09ab"
server_key_version: "encryptionkey@0"
I20260809 04:44:40.467615 30163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:40.496385 30163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:40.499838 30163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:40.500017 30163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:40.507859 30163 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:39553
I20260809 04:44:40.507891 30232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:39553 every 8 connection(s)
I20260809 04:44:40.508893 30163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:40.511709 30233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:40.512223 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 30163
I20260809 04:44:40.512367 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:40.512709 6011 external_mini_cluster.cc:1428] Setting key 2fdebebadc5f3320b700a8aa071e4dee
I20260809 04:44:40.518443 30233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166: Bootstrap starting.
I20260809 04:44:40.520622 30233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:40.521477 30233 log.cc:824] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:40.522138 30237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:40.514200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55917 (local address 127.5.222.254:39553)
0809 04:44:40.514646 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:40.514662 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:40.514744 (+ 82us) server_negotiation.cc:407] Connection header received
0809 04:44:40.515437 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:40.515454 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:40.515797 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:40.516151 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:40.516568 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.517625 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.518567 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.519054 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.519268 (+ 214us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:40.520285 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:40.520291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:40.520304 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:40.520704 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:40.520798 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:40.520836 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:40.520983 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:40.521212 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 04:44:40.521358 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":134,"threads_started":1}
I20260809 04:44:40.523546 30233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166: No bootstrap required, opened a new log
I20260809 04:44:40.525419 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:40.527740 30233 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba54494683b470b972ec2f2ae210166" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39553 } }
I20260809 04:44:40.528033 30233 raft_consensus.cc:374] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:40.528095 30233 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba54494683b470b972ec2f2ae210166, State: Initialized, Role: FOLLOWER
I20260809 04:44:40.528584 30233 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba54494683b470b972ec2f2ae210166" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39553 } }
I20260809 04:44:40.528776 30233 raft_consensus.cc:388] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:40.528847 30233 raft_consensus.cc:482] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:40.528950 30233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:40.530063 30233 raft_consensus.cc:504] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba54494683b470b972ec2f2ae210166" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39553 } }
I20260809 04:44:40.530411 30233 leader_election.cc:302] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eba54494683b470b972ec2f2ae210166; no voters:
I20260809 04:44:40.530747 30233 leader_election.cc:288] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:40.530833 30244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:40.531045 30244 raft_consensus.cc:686] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [term 1 LEADER]: Becoming Leader. State: Replica: eba54494683b470b972ec2f2ae210166, State: Running, Role: LEADER
I20260809 04:44:40.531378 30244 consensus_queue.cc:237] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba54494683b470b972ec2f2ae210166" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39553 } }
I20260809 04:44:40.532171 30237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:40.527699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46963 (local address 127.5.222.254:39553)
0809 04:44:40.527811 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:40.527819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:40.528044 (+ 225us) server_negotiation.cc:407] Connection header received
0809 04:44:40.528235 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:40.528239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:40.528290 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:40.528377 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:40.528651 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.529454 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.530285 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.530578 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.530674 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:40.531568 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:40.531571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:40.531574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:40.531586 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:40.531613 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:40.531625 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:40.531704 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:40.532014 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 04:44:40.532053 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 04:44:40.532177 30233 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:40.533802 30246 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eba54494683b470b972ec2f2ae210166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba54494683b470b972ec2f2ae210166" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39553 } } }
I20260809 04:44:40.534183 30246 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:40.534080 30245 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba54494683b470b972ec2f2ae210166. Latest consensus state: current_term: 1 leader_uuid: "eba54494683b470b972ec2f2ae210166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba54494683b470b972ec2f2ae210166" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 39553 } } }
I20260809 04:44:40.534778 30245 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166 [sys.catalog]: This master's current role is: LEADER
W20260809 04:44:40.537236 30258 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:40.537321 30258 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:40.537415 30259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:40.538390 30259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:40.544396 30259 catalog_manager.cc:1347] Generated new cluster ID: a4b61bbd43dd4676880fe98a71ac53c8
I20260809 04:44:40.544477 30259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:40.563753 30259 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:40.564821 30259 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:40.570238 30259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eba54494683b470b972ec2f2ae210166: Generated new TSK 0
I20260809 04:44:40.570955 30259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:40.575237 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 30163
W20260809 04:44:40.582419 30145 connection.cc:535] client connection to 127.5.222.254:39553 recv error: Network error: failed to read from TLS socket (remote: 127.5.222.254:39553): Connection reset by peer (error 104)
2026-08-09T04:44:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1511 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:40 dist-test-slave-vxj2 krb5kdc[30266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:40 dist-test-slave-vxj2 krb5kdc[30266](info): set up 2 sockets
Aug 09 04:44:40 dist-test-slave-vxj2 krb5kdc[30266](info): commencing operation
krb5kdc: starting...
W20260809 04:44:41.793124 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:41 dist-test-slave-vxj2 krb5kdc[30266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:44:41.932498 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:43873
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:43873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:42.044776 30279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:42.045074 30279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:42.045163 30279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:42.049118 30279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:42.049233 30279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:42.049274 30279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:42.049327 30279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:42.049367 30279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:42.054617 30279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:43873
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:43873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:42.055687 30279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:42.056691 30279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:42.064975 30285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:42.065002 30284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:42.064977 30287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:42.065342 30279 server_base.cc:1034] running on GCE node
I20260809 04:44:42.065868 30279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:42.066951 30279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:42.068109 30279 hybrid_clock.cc:648] HybridClock initialized: now 1786250682068089 us; error 31 us; skew 500 ppm
Aug 09 04:44:42 dist-test-slave-vxj2 krb5kdc[30266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250682, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:42.070881 30279 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:44:42.071745 30279 webserver.cc:460] Webserver started at http://127.5.222.254:41487/ using document root <none> and password file <none>
I20260809 04:44:42.072283 30279 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:42.072367 30279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:42.072589 30279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:42.074643 30279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "1932a6af550349a9be8b794a779864c7"
format_stamp: "Formatted at 2026-08-09 04:44:42 on dist-test-slave-vxj2"
server_key: "fa06153f7c6f8a12d2ba8148afddd93c"
server_key_iv: "cc4ec4b7090169c96a73691457df9880"
server_key_version: "encryptionkey@0"
I20260809 04:44:42.075203 30279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "1932a6af550349a9be8b794a779864c7"
format_stamp: "Formatted at 2026-08-09 04:44:42 on dist-test-slave-vxj2"
server_key: "fa06153f7c6f8a12d2ba8148afddd93c"
server_key_iv: "cc4ec4b7090169c96a73691457df9880"
server_key_version: "encryptionkey@0"
I20260809 04:44:42.078924 30279 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:44:42.081626 30294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:42.082882 30279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:42.083033 30279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "1932a6af550349a9be8b794a779864c7"
format_stamp: "Formatted at 2026-08-09 04:44:42 on dist-test-slave-vxj2"
server_key: "fa06153f7c6f8a12d2ba8148afddd93c"
server_key_iv: "cc4ec4b7090169c96a73691457df9880"
server_key_version: "encryptionkey@0"
I20260809 04:44:42.083153 30279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:42.088162 30279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:42.091168 30279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:42.091390 30279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:42.099316 30279 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:43873
I20260809 04:44:42.099359 30349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:43873 every 8 connection(s)
I20260809 04:44:42.100307 30279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:42.103088 30350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:42.108454 30350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7: Bootstrap starting.
I20260809 04:44:42.108886 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 30279
I20260809 04:44:42.109050 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:42.109395 6011 external_mini_cluster.cc:1428] Setting key d02c3f155645a038f890ab6285f7f316
I20260809 04:44:42.111212 30350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:42.112092 30350 log.cc:824] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:42.114351 30350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7: No bootstrap required, opened a new log
I20260809 04:44:42.115118 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:42.110807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34593 (local address 127.5.222.254:43873)
0809 04:44:42.111287 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:42.111305 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:42.111349 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:44:42.112211 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:42.112234 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:42.112590 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:42.112906 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:42.113741 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:42.113747 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:42.113761 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:42.113772 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:42.113838 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:42.113864 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:42.113972 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:42.114094 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 04:44:42.114241 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":129,"threads_started":1}
I20260809 04:44:42.117807 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:42.118153 30350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1932a6af550349a9be8b794a779864c7" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43873 } }
I20260809 04:44:42.118381 30350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:42.118459 30350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1932a6af550349a9be8b794a779864c7, State: Initialized, Role: FOLLOWER
I20260809 04:44:42.118860 30350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1932a6af550349a9be8b794a779864c7" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43873 } }
I20260809 04:44:42.119014 30350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:42.119084 30350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:42.119196 30350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:42.120162 30350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1932a6af550349a9be8b794a779864c7" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43873 } }
I20260809 04:44:42.120509 30350 leader_election.cc:302] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1932a6af550349a9be8b794a779864c7; no voters:
I20260809 04:44:42.120831 30350 leader_election.cc:288] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:42.120895 30360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:42.121114 30360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [term 1 LEADER]: Becoming Leader. State: Replica: 1932a6af550349a9be8b794a779864c7, State: Running, Role: LEADER
I20260809 04:44:42.121466 30360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1932a6af550349a9be8b794a779864c7" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43873 } }
I20260809 04:44:42.121797 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:42.119987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54755 (local address 127.5.222.254:43873)
0809 04:44:42.120107 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:42.120111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:42.120334 (+ 223us) server_negotiation.cc:407] Connection header received
0809 04:44:42.120460 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:42.120462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:42.120523 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:42.120600 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:42.121378 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:42.121383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:42.121386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:42.121400 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:42.121427 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:42.121442 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:42.121499 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:42.121659 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 04:44:42.121695 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:44:42.122169 30350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:42.123356 30361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1932a6af550349a9be8b794a779864c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1932a6af550349a9be8b794a779864c7" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43873 } } }
I20260809 04:44:42.123507 30361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:42.123906 30370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:42.123870 30362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1932a6af550349a9be8b794a779864c7. Latest consensus state: current_term: 1 leader_uuid: "1932a6af550349a9be8b794a779864c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1932a6af550349a9be8b794a779864c7" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 43873 } } }
I20260809 04:44:42.124069 30362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:42.126859 30370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:42.132238 30370 catalog_manager.cc:1347] Generated new cluster ID: 18f9517d302e4ad2872f2b09b0a213f0
I20260809 04:44:42.132310 30370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:42.144032 30370 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:42.144332 30370 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:42.150223 30370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1932a6af550349a9be8b794a779864c7: Generated new TSK 0
I20260809 04:44:42.150836 30370 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:42.157027 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 30279
2026-08-09T04:44:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1582 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:42 dist-test-slave-vxj2 krb5kdc[30382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:42 dist-test-slave-vxj2 krb5kdc[30382](info): set up 2 sockets
Aug 09 04:44:42 dist-test-slave-vxj2 krb5kdc[30382](info): commencing operation
krb5kdc: starting...
W20260809 04:44:43.252108 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:43 dist-test-slave-vxj2 krb5kdc[30382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:44:43.375860 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:38439
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:43691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:38439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:43.489670 30395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:43.489965 30395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:43.490047 30395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:43.494814 30395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:43.494891 30395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:43.494915 30395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:43.494959 30395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:43.495007 30395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:43.500576 30395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:43691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:38439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:38439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:43.501803 30395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:43.502866 30395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:43.512120 30400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:43.512152 30401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:43.512238 30395 server_base.cc:1034] running on GCE node
W20260809 04:44:43.512179 30405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:43.512890 30395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:43.514092 30395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:43.515301 30395 hybrid_clock.cc:648] HybridClock initialized: now 1786250683515274 us; error 38 us; skew 500 ppm
Aug 09 04:44:43 dist-test-slave-vxj2 krb5kdc[30382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250683, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:43.518428 30395 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:44:43.519287 30395 webserver.cc:460] Webserver started at http://127.5.222.254:43759/ using document root <none> and password file <none>
I20260809 04:44:43.519830 30395 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:43.519891 30395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:43.520052 30395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:43.522152 30395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "14f3eb850e1d4483aed2aa2bb12ab426"
format_stamp: "Formatted at 2026-08-09 04:44:43 on dist-test-slave-vxj2"
server_key: "ee33a4fa699c18ae96fb0c6b2301ac3a"
server_key_iv: "42cf9abd849313b17cf0b7d6fca5872d"
server_key_version: "encryptionkey@0"
I20260809 04:44:43.522724 30395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "14f3eb850e1d4483aed2aa2bb12ab426"
format_stamp: "Formatted at 2026-08-09 04:44:43 on dist-test-slave-vxj2"
server_key: "ee33a4fa699c18ae96fb0c6b2301ac3a"
server_key_iv: "42cf9abd849313b17cf0b7d6fca5872d"
server_key_version: "encryptionkey@0"
I20260809 04:44:43.526384 30395 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:44:43.528888 30414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:43.530195 30395 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:43.530328 30395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "14f3eb850e1d4483aed2aa2bb12ab426"
format_stamp: "Formatted at 2026-08-09 04:44:43 on dist-test-slave-vxj2"
server_key: "ee33a4fa699c18ae96fb0c6b2301ac3a"
server_key_iv: "42cf9abd849313b17cf0b7d6fca5872d"
server_key_version: "encryptionkey@0"
I20260809 04:44:43.530453 30395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:43.546844 30395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:43.549947 30395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:43.550166 30395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:43.557593 30395 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:38439
I20260809 04:44:43.557662 30473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:38439 every 8 connection(s)
I20260809 04:44:43.558631 30395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:43.561852 30474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:43.562465 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 30395
I20260809 04:44:43.562644 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:43.562965 6011 external_mini_cluster.cc:1428] Setting key c4198ed043b63284bcd12641092b8610
I20260809 04:44:43.569263 30474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426: Bootstrap starting.
I20260809 04:44:43.571966 30474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:43.572764 30474 log.cc:824] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:44:43 dist-test-slave-vxj2 krb5kdc[30382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:43.575064 30474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426: No bootstrap required, opened a new log
I20260809 04:44:43.578233 30477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:43.564483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51325 (local address 127.5.222.254:38439)
0809 04:44:43.564959 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:43.564980 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:43.565025 (+ 45us) server_negotiation.cc:407] Connection header received
0809 04:44:43.565923 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:43.565949 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:43.566470 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:43.566847 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:43.567548 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.569056 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.570018 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.570727 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.570892 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:43.573596 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:43.573602 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:43.573620 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:43.573655 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:43.575346 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:43.575845 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:43.575850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:43.575854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:43.575968 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:43.576265 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:43.576267 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:43.576268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:43.576656 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:43.576811 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:43.577131 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 04:44:43.577483 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":114,"threads_started":1}
I20260809 04:44:43.579296 30474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f3eb850e1d4483aed2aa2bb12ab426" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38439 } }
I20260809 04:44:43.579545 30474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:43.579624 30474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f3eb850e1d4483aed2aa2bb12ab426, State: Initialized, Role: FOLLOWER
I20260809 04:44:43.580080 30474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f3eb850e1d4483aed2aa2bb12ab426" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38439 } }
I20260809 04:44:43.580260 30474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:43.580363 30474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:43.580466 30474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:43.581400 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:43.581535 30474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f3eb850e1d4483aed2aa2bb12ab426" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38439 } }
I20260809 04:44:43.581889 30474 leader_election.cc:302] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14f3eb850e1d4483aed2aa2bb12ab426; no voters:
I20260809 04:44:43.582239 30474 leader_election.cc:288] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:43.582372 30479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:43.582647 30479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [term 1 LEADER]: Becoming Leader. State: Replica: 14f3eb850e1d4483aed2aa2bb12ab426, State: Running, Role: LEADER
I20260809 04:44:43.583012 30479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f3eb850e1d4483aed2aa2bb12ab426" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38439 } }
I20260809 04:44:43.583616 30474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:43.584221 30480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14f3eb850e1d4483aed2aa2bb12ab426. Latest consensus state: current_term: 1 leader_uuid: "14f3eb850e1d4483aed2aa2bb12ab426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f3eb850e1d4483aed2aa2bb12ab426" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38439 } } }
I20260809 04:44:43.584405 30480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:43.584647 30481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14f3eb850e1d4483aed2aa2bb12ab426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f3eb850e1d4483aed2aa2bb12ab426" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 38439 } } }
I20260809 04:44:43.584760 30481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:43.585418 30491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:43.588640 30491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:43.594509 30477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:43.583732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43293 (local address 127.5.222.254:38439)
0809 04:44:43.583841 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:43.583845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:43.585921 (+ 2076us) server_negotiation.cc:407] Connection header received
0809 04:44:43.586113 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:43.586117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:43.586166 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:43.586254 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:43.587031 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.587831 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.588973 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.589375 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.589489 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:43.591110 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:43.591114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:43.591116 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:43.591141 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:43.592638 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:43.593137 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:43.593141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:43.593143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:43.593255 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:43.593586 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:43.593590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:43.593592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:43.593830 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:43.593968 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:43.594241 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 04:44:43.594361 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 04:44:43.595732 30491 catalog_manager.cc:1347] Generated new cluster ID: 63e81cf8275d4bc7a05ec618f6bc554a
I20260809 04:44:43.595796 30491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:43.620074 30491 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:43.621333 30491 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:43.627791 30491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14f3eb850e1d4483aed2aa2bb12ab426: Generated new TSK 0
I20260809 04:44:43.628494 30491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:43.639744 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 30395
2026-08-09T04:44:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1482 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:43 dist-test-slave-vxj2 krb5kdc[30508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:43 dist-test-slave-vxj2 krb5kdc[30508](info): set up 2 sockets
Aug 09 04:44:43 dist-test-slave-vxj2 krb5kdc[30508](info): commencing operation
krb5kdc: starting...
W20260809 04:44:44.749433 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:44 dist-test-slave-vxj2 krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:44:44.880419 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:36729
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:36729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:44.991317 30523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:44.991637 30523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:44.991709 30523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:44.995748 30523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:44.995857 30523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:44.995899 30523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:44.995939 30523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:44.995973 30523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:45.001170 30523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:36729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:36729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:45.002256 30523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:45.003296 30523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:45.011853 30529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:45.011958 30528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:45.011885 30531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:45.012286 30523 server_base.cc:1034] running on GCE node
I20260809 04:44:45.012813 30523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:45.014016 30523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:45.015209 30523 hybrid_clock.cc:648] HybridClock initialized: now 1786250685015180 us; error 45 us; skew 500 ppm
Aug 09 04:44:45 dist-test-slave-vxj2 krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250685, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:45.018047 30523 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:44:45.019003 30523 webserver.cc:460] Webserver started at http://127.5.222.254:45487/ using document root <none> and password file <none>
I20260809 04:44:45.019598 30523 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:45.019696 30523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:45.019943 30523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:45.022080 30523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "d47711332e734abe99584a2331bea29f"
format_stamp: "Formatted at 2026-08-09 04:44:45 on dist-test-slave-vxj2"
server_key: "68227675465cd72f983004e41667aee7"
server_key_iv: "324c19dea0914322d4e7407a2c351f5d"
server_key_version: "encryptionkey@0"
I20260809 04:44:45.022660 30523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "d47711332e734abe99584a2331bea29f"
format_stamp: "Formatted at 2026-08-09 04:44:45 on dist-test-slave-vxj2"
server_key: "68227675465cd72f983004e41667aee7"
server_key_iv: "324c19dea0914322d4e7407a2c351f5d"
server_key_version: "encryptionkey@0"
I20260809 04:44:45.026379 30523 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:44:45.028712 30538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:45.030054 30523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:45.030203 30523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "d47711332e734abe99584a2331bea29f"
format_stamp: "Formatted at 2026-08-09 04:44:45 on dist-test-slave-vxj2"
server_key: "68227675465cd72f983004e41667aee7"
server_key_iv: "324c19dea0914322d4e7407a2c351f5d"
server_key_version: "encryptionkey@0"
I20260809 04:44:45.030315 30523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:45.064590 30523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:45.068313 30523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:45.068543 30523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:45.075991 30523 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:36729
I20260809 04:44:45.076020 30594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:36729 every 8 connection(s)
I20260809 04:44:45.076939 30523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:45.077203 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 30523
I20260809 04:44:45.077359 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:45.077698 6011 external_mini_cluster.cc:1428] Setting key 42085c5f6c76fd05b21a2ece3c4d84cd
I20260809 04:44:45.080137 30597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:45.085861 30597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f: Bootstrap starting.
I20260809 04:44:45.086627 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:45.079181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60273 (local address 127.5.222.254:36729)
0809 04:44:45.079627 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:45.079641 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:45.079675 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:44:45.080374 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:45.080391 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:45.080722 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:45.081053 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:45.081493 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.082470 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.083359 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.083841 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.084015 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:45.084886 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:45.084890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:45.084902 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:45.085341 (+ 439us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:45.085397 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:45.085421 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:45.085543 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:45.085671 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 04:44:45.085873 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":147,"threads_started":1}
I20260809 04:44:45.088330 30597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:45.089250 30597 log.cc:824] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:45.089561 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:45.091261 30597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f: No bootstrap required, opened a new log
I20260809 04:44:45.095827 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:45.091677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46711 (local address 127.5.222.254:36729)
0809 04:44:45.091811 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:45.091815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:45.092015 (+ 200us) server_negotiation.cc:407] Connection header received
0809 04:44:45.092123 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:45.092125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:45.092161 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:45.092227 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:45.092517 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.093061 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.094298 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.094509 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.094586 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:45.095353 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:45.095356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:45.095358 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:45.095368 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:45.095391 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:45.095401 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:45.095459 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:45.095588 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 04:44:45.095630 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:44:45.095872 30597 raft_consensus.cc:348] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47711332e734abe99584a2331bea29f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36729 } }
I20260809 04:44:45.096215 30597 raft_consensus.cc:374] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:45.096277 30597 raft_consensus.cc:729] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d47711332e734abe99584a2331bea29f, State: Initialized, Role: FOLLOWER
I20260809 04:44:45.096724 30597 consensus_queue.cc:260] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47711332e734abe99584a2331bea29f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36729 } }
I20260809 04:44:45.096879 30597 raft_consensus.cc:388] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:45.096951 30597 raft_consensus.cc:482] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:45.097045 30597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:45.097993 30597 raft_consensus.cc:504] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47711332e734abe99584a2331bea29f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36729 } }
I20260809 04:44:45.098359 30597 leader_election.cc:302] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d47711332e734abe99584a2331bea29f; no voters:
I20260809 04:44:45.100028 30597 leader_election.cc:288] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:45.100149 30608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:45.100407 30608 raft_consensus.cc:686] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [term 1 LEADER]: Becoming Leader. State: Replica: d47711332e734abe99584a2331bea29f, State: Running, Role: LEADER
I20260809 04:44:45.100775 30608 consensus_queue.cc:237] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47711332e734abe99584a2331bea29f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36729 } }
I20260809 04:44:45.101333 30597 sys_catalog.cc:564] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:45.102581 30610 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d47711332e734abe99584a2331bea29f. Latest consensus state: current_term: 1 leader_uuid: "d47711332e734abe99584a2331bea29f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47711332e734abe99584a2331bea29f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36729 } } }
I20260809 04:44:45.102607 30609 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d47711332e734abe99584a2331bea29f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47711332e734abe99584a2331bea29f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 36729 } } }
I20260809 04:44:45.102703 30610 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:45.102720 30609 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:45.103304 30617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:45.106233 30617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:45.111763 30617 catalog_manager.cc:1347] Generated new cluster ID: ce3011b6997d4de58ba4584b3f45a31d
I20260809 04:44:45.111831 30617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:45.134830 30617 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:45.136088 30617 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:45.144179 30617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d47711332e734abe99584a2331bea29f: Generated new TSK 0
I20260809 04:44:45.144940 30617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:45.157445 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 30523
2026-08-09T04:44:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1519 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:45 dist-test-slave-vxj2 krb5kdc[30629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:45 dist-test-slave-vxj2 krb5kdc[30629](info): set up 2 sockets
Aug 09 04:44:45 dist-test-slave-vxj2 krb5kdc[30629](info): commencing operation
krb5kdc: starting...
W20260809 04:44:46.256983 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:46 dist-test-slave-vxj2 krb5kdc[30629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:44:46.380852 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:34761
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:37781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:34761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:46.491117 30642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:46.491379 30642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:46.491420 30642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:46.495368 30642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:46.495425 30642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:46.495445 30642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:46.495461 30642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:46.495476 30642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:46.500532 30642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:37781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:34761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:34761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:46.501488 30642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:46.502419 30642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:46.510612 30647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:46.510651 30648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:46.510664 30651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:46.510725 30642 server_base.cc:1034] running on GCE node
I20260809 04:44:46.511415 30642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:46.512452 30642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:46.513612 30642 hybrid_clock.cc:648] HybridClock initialized: now 1786250686513584 us; error 38 us; skew 500 ppm
Aug 09 04:44:46 dist-test-slave-vxj2 krb5kdc[30629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250686, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:46.516512 30642 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:44:46.517474 30642 webserver.cc:460] Webserver started at http://127.5.222.254:46097/ using document root <none> and password file <none>
I20260809 04:44:46.518011 30642 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:46.518098 30642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:46.518332 30642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:46.520298 30642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "156553ec49a14417b8da68aefb850a42"
format_stamp: "Formatted at 2026-08-09 04:44:46 on dist-test-slave-vxj2"
server_key: "ff33952ce81e6d2598f344729fe5d5ac"
server_key_iv: "9326dabcee326dac64b9f38ca9aa23b9"
server_key_version: "encryptionkey@0"
I20260809 04:44:46.520859 30642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "156553ec49a14417b8da68aefb850a42"
format_stamp: "Formatted at 2026-08-09 04:44:46 on dist-test-slave-vxj2"
server_key: "ff33952ce81e6d2598f344729fe5d5ac"
server_key_iv: "9326dabcee326dac64b9f38ca9aa23b9"
server_key_version: "encryptionkey@0"
I20260809 04:44:46.524402 30642 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 04:44:46.526836 30658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:46.528025 30642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:44:46.528180 30642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "156553ec49a14417b8da68aefb850a42"
format_stamp: "Formatted at 2026-08-09 04:44:46 on dist-test-slave-vxj2"
server_key: "ff33952ce81e6d2598f344729fe5d5ac"
server_key_iv: "9326dabcee326dac64b9f38ca9aa23b9"
server_key_version: "encryptionkey@0"
I20260809 04:44:46.528304 30642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:46.532838 30642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:46.535689 30642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:46.535881 30642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:46.542929 30642 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:34761
I20260809 04:44:46.542956 30717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:34761 every 8 connection(s)
I20260809 04:44:46.543828 30642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:46.546558 30718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:46.546638 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 30642
I20260809 04:44:46.546762 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:46.547078 6011 external_mini_cluster.cc:1428] Setting key d519bf06c234470fb2d96e58b5cfff86
I20260809 04:44:46.552467 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:46.548521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33897 (local address 127.5.222.254:34761)
0809 04:44:46.549030 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:46.549044 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:46.549082 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:44:46.549888 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:46.549922 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:46.550190 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:46.550527 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:46.551344 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:46.551348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:46.551360 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:46.551370 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:46.551426 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:46.551444 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:46.551597 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:46.551664 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 04:44:46.551823 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":142,"threads_started":1}
I20260809 04:44:46.553383 30718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42: Bootstrap starting.
I20260809 04:44:46.555236 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:46.555828 30718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:46.556607 30718 log.cc:824] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:46.559101 30718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42: No bootstrap required, opened a new log
I20260809 04:44:46.559302 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:46.557411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42287 (local address 127.5.222.254:34761)
0809 04:44:46.557531 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:46.557536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:46.557740 (+ 204us) server_negotiation.cc:407] Connection header received
0809 04:44:46.557899 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:46.557904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:46.557947 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:46.558043 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:46.558854 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:46.558857 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:46.558860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:46.558872 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:46.558903 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:46.558917 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:46.559002 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:46.559111 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 04:44:46.559143 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:44:46.563438 30718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "156553ec49a14417b8da68aefb850a42" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 34761 } }
I20260809 04:44:46.563660 30718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:46.563730 30718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 156553ec49a14417b8da68aefb850a42, State: Initialized, Role: FOLLOWER
I20260809 04:44:46.564181 30718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "156553ec49a14417b8da68aefb850a42" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 34761 } }
I20260809 04:44:46.564330 30718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:46.564400 30718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:46.564502 30718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:46.565601 30718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "156553ec49a14417b8da68aefb850a42" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 34761 } }
I20260809 04:44:46.565944 30718 leader_election.cc:302] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 156553ec49a14417b8da68aefb850a42; no voters:
I20260809 04:44:46.566289 30718 leader_election.cc:288] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:46.566439 30728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:46.566690 30728 raft_consensus.cc:686] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [term 1 LEADER]: Becoming Leader. State: Replica: 156553ec49a14417b8da68aefb850a42, State: Running, Role: LEADER
I20260809 04:44:46.567065 30728 consensus_queue.cc:237] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "156553ec49a14417b8da68aefb850a42" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 34761 } }
I20260809 04:44:46.567615 30718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:46.569380 30730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 156553ec49a14417b8da68aefb850a42. Latest consensus state: current_term: 1 leader_uuid: "156553ec49a14417b8da68aefb850a42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "156553ec49a14417b8da68aefb850a42" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 34761 } } }
I20260809 04:44:46.569725 30730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:46.569610 30729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "156553ec49a14417b8da68aefb850a42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "156553ec49a14417b8da68aefb850a42" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 34761 } } }
I20260809 04:44:46.570164 30729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42 [sys.catalog]: This master's current role is: LEADER
W20260809 04:44:46.572378 30746 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:46.572441 30746 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:46.572520 30745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:46.573257 30745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:46.578547 30745 catalog_manager.cc:1347] Generated new cluster ID: 7032e8a6c61e45c0b2da3ba098cc6328
I20260809 04:44:46.578619 30745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:46.590953 30745 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:46.591282 30745 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:46.600607 30745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 156553ec49a14417b8da68aefb850a42: Generated new TSK 0
I20260809 04:44:46.601338 30745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:46.612666 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 30642
2026-08-09T04:44:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1455 ms)
[----------] 18 tests from AuthTokenIssuingTest (26987 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:46 dist-test-slave-vxj2 krb5kdc[30751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:46 dist-test-slave-vxj2 krb5kdc[30751](info): set up 2 sockets
Aug 09 04:44:46 dist-test-slave-vxj2 krb5kdc[30751](info): commencing operation
krb5kdc: starting...
W20260809 04:44:47.732792 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:47 dist-test-slave-vxj2 krb5kdc[30751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:44:47.858386 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:45207
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:45207,127.5.222.253:33989,127.5.222.252:33785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:47.967406 30764 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:44:47.967726 30764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:47.967800 30764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:47.967880 30764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:47.971906 30764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:47.971988 30764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:47.972025 30764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:47.972061 30764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:47.972095 30764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:47.977443 30764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:45207,127.5.222.253:33989,127.5.222.252:33785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:45207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:47.978519 30764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:47.979534 30764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:47.988224 30770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:47.988199 30772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:47.988266 30764 server_base.cc:1034] running on GCE node
W20260809 04:44:47.988266 30769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:47.988993 30764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:47.990167 30764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:47.991346 30764 hybrid_clock.cc:648] HybridClock initialized: now 1786250687991309 us; error 51 us; skew 500 ppm
Aug 09 04:44:47 dist-test-slave-vxj2 krb5kdc[30751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250687, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:47.998493 30764 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:44:47.999362 30764 webserver.cc:460] Webserver started at http://127.5.222.254:34139/ using document root <none> and password file <none>
I20260809 04:44:47.999871 30764 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:47.999927 30764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:48.000088 30764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:48.002096 30764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "277351e526d74a21b7473618b736048f"
format_stamp: "Formatted at 2026-08-09 04:44:47 on dist-test-slave-vxj2"
server_key: "65746ccfcf81c685290b1d7155582f4e"
server_key_iv: "a0f632b76bf6537dc6f4b13ad3746b87"
server_key_version: "encryptionkey@0"
I20260809 04:44:48.002602 30764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "277351e526d74a21b7473618b736048f"
format_stamp: "Formatted at 2026-08-09 04:44:47 on dist-test-slave-vxj2"
server_key: "65746ccfcf81c685290b1d7155582f4e"
server_key_iv: "a0f632b76bf6537dc6f4b13ad3746b87"
server_key_version: "encryptionkey@0"
I20260809 04:44:48.006341 30764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 04:44:48.008728 30779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:48.010113 30764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:44:48.010264 30764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "277351e526d74a21b7473618b736048f"
format_stamp: "Formatted at 2026-08-09 04:44:47 on dist-test-slave-vxj2"
server_key: "65746ccfcf81c685290b1d7155582f4e"
server_key_iv: "a0f632b76bf6537dc6f4b13ad3746b87"
server_key_version: "encryptionkey@0"
I20260809 04:44:48.010378 30764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:48.024501 30764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:48.027323 30764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:48.027530 30764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:48.034807 30764 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:45207
I20260809 04:44:48.035068 30839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:45207 every 8 connection(s)
I20260809 04:44:48.035881 30764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:48.038625 30840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:48.042303 30840 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:48.044595 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 30764
I20260809 04:44:48.044767 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:48.045061 6011 external_mini_cluster.cc:1428] Setting key 4f5e46e5e5abecaf0321375b7f720564
WARNING: no policy specified for kudu/127.5.222.253@KRBTEST.COM; defaulting to no policy
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250687, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.253@KRBTEST.COM" created.
I20260809 04:44:48.059875 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.044598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:45207 (local address 127.0.0.1:53212)
0809 04:44:48.045643 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.045682 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:48.046638 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:48.047383 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:48.047391 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:48.047898 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:48.048227 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.048243 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.050183 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.050186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.050954 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.050964 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.051919 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.051922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.052050 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.052697 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:48.052712 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:48.054536 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:48.056575 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.056584 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.056591 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.056946 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.057325 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.057327 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.057329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.057445 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.058242 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:48.058247 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:48.058364 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 04:44:48.058543 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 04:44:48.058902 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"mutex_wait_us":151,"thread_start_us":146,"threads_started":1}
I20260809 04:44:48.059875 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.044598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53212 (local address 127.5.222.254:45207)
0809 04:44:48.045191 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.045212 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.045800 (+ 588us) server_negotiation.cc:407] Connection header received
0809 04:44:48.046981 (+ 1181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.046985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.047155 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.047322 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.048417 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.050036 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.051101 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.051751 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.051936 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.054684 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.054690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.054699 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.054720 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.056473 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.057050 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.057055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.057056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.057214 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.057545 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.057551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.057556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.058122 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.058268 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.058604 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.058849 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":434,"spinlock_wait_cycles":48896,"thread_start_us":158,"threads_started":1}
I20260809 04:44:48.062600 30840 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:48.063424 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.063108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:33989 (local address 127.0.0.1:39168)
0809 04:44:48.063259 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.063343 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:33989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 04:44:48.063648 30781 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.222.253:33989: connect: Connection refused (error 111)
I20260809 04:44:48.064508 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.064239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:33989 (local address 127.0.0.1:39182)
0809 04:44:48.064364 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.064428 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:33989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:44:48.064687 30840 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.253:33989: Network error: Client connection negotiation failed: client connection to 127.5.222.253:33989: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.253.keytab.
Entry for principal kudu/127.5.222.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.253.keytab.
I20260809 04:44:48.074159 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.253:33989
--webserver_interface=127.5.222.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.253.keytab
--principal=kudu/127.5.222.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:45207,127.5.222.253:33989,127.5.222.252:33785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:44:48.113982 30840 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } attempt: 1
I20260809 04:44:48.115031 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.114640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:33989 (local address 127.0.0.1:39188)
0809 04:44:48.114829 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.114924 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:33989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 04:44:48.115828 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.115572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:33989 (local address 127.0.0.1:39204)
0809 04:44:48.115694 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.115755 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:33989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 04:44:48.116008 30840 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.253:33989: Network error: Client connection negotiation failed: client connection to 127.5.222.253:33989: connect: Connection refused (error 111)
I20260809 04:44:48.184248 30840 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } attempt: 2
I20260809 04:44:48.185209 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.184800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:33989 (local address 127.0.0.1:39206)
0809 04:44:48.184984 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.185061 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:33989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 04:44:48.185956 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.185718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:33989 (local address 127.0.0.1:39210)
0809 04:44:48.185832 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.185887 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:33989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 04:44:48.186098 30840 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.253:33989: Network error: Client connection negotiation failed: client connection to 127.5.222.253:33989: connect: Connection refused (error 111)
W20260809 04:44:48.187731 30847 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:44:48.188014 30847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:48.188068 30847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:48.188186 30847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:48.192286 30847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:48.192350 30847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:48.192401 30847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:48.192441 30847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:48.192473 30847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:48.197737 30847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:45207,127.5.222.253:33989,127.5.222.252:33785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.253.keytab
--principal=kudu/127.5.222.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.253:33989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.222.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:48.198736 30847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:48.199808 30847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:48.208257 30856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:48.208303 30853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:48.208274 30852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:48.208523 30847 server_base.cc:1034] running on GCE node
I20260809 04:44:48.209198 30847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:48.210366 30847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:48.211586 30847 hybrid_clock.cc:648] HybridClock initialized: now 1786250688211560 us; error 32 us; skew 500 ppm
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:48.214851 30847 init.cc:377] Logged in from keytab as kudu/127.5.222.253@KRBTEST.COM (short username kudu)
I20260809 04:44:48.215884 30847 webserver.cc:460] Webserver started at http://127.5.222.253:42057/ using document root <none> and password file <none>
I20260809 04:44:48.216480 30847 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:48.216578 30847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:48.216840 30847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:48.219076 30847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/data/instance:
uuid: "773f4d4e93484a70b4941054e9c4b342"
format_stamp: "Formatted at 2026-08-09 04:44:48 on dist-test-slave-vxj2"
server_key: "9b247346cf9f48058f5d20382dcdbfaf"
server_key_iv: "324df759562703e7adbbddf6d8ac7344"
server_key_version: "encryptionkey@0"
I20260809 04:44:48.219683 30847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/wal/instance:
uuid: "773f4d4e93484a70b4941054e9c4b342"
format_stamp: "Formatted at 2026-08-09 04:44:48 on dist-test-slave-vxj2"
server_key: "9b247346cf9f48058f5d20382dcdbfaf"
server_key_iv: "324df759562703e7adbbddf6d8ac7344"
server_key_version: "encryptionkey@0"
I20260809 04:44:48.223675 30847 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:48.226257 30863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:48.227484 30847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:44:48.227636 30847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/wal
uuid: "773f4d4e93484a70b4941054e9c4b342"
format_stamp: "Formatted at 2026-08-09 04:44:48 on dist-test-slave-vxj2"
server_key: "9b247346cf9f48058f5d20382dcdbfaf"
server_key_iv: "324df759562703e7adbbddf6d8ac7344"
server_key_version: "encryptionkey@0"
I20260809 04:44:48.227783 30847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:48.242079 30847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:48.245138 30847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:48.245436 30847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:48.252713 30847 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.253:33989
I20260809 04:44:48.252990 30919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.253:33989 every 8 connection(s)
I20260809 04:44:48.253754 30847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
I20260809 04:44:48.256408 30920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:48.260037 30920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:48.260668 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 30847
I20260809 04:44:48.260800 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-1/wal/instance
I20260809 04:44:48.261101 6011 external_mini_cluster.cc:1428] Setting key b10e596ce5b5622fa5770a1207e79585
WARNING: no policy specified for kudu/127.5.222.252@KRBTEST.COM; defaulting to no policy
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.253@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.252@KRBTEST.COM" created.
I20260809 04:44:48.274515 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.262266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.5.222.254:45207)
0809 04:44:48.262448 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.262454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.263410 (+ 956us) server_negotiation.cc:407] Connection header received
0809 04:44:48.264323 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.264326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.264389 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.264495 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.265761 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.266308 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.267640 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.267882 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.268004 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.270433 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.270438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.270441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.270465 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.272023 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.272541 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.272547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.272549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.272666 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.273080 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.273083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.273085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.273359 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.273488 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.274094 (+ 606us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.274255 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:44:48.275494 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.262734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:45207 (local address 127.0.0.1:53214)
0809 04:44:48.263261 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.263303 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:48.264120 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:48.264658 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:48.264665 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:48.264993 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:48.265614 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.265630 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.266465 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.266469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.267502 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.267512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.268043 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.268048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.268216 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.268806 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:48.268822 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:48.270256 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:48.272145 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.272150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.272165 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.272422 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.272786 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.272792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.272793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.272973 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.273490 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:48.273496 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:48.273788 (+ 292us) client_negotiation.cc:769] Sending connection context
0809 04:44:48.273985 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 04:44:48.274266 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":156,"threads_started":1}
I20260809 04:44:48.276914 30920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:48.277204 30840 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } attempt: 3
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.253@KRBTEST.COM for kudu/127.5.222.253@KRBTEST.COM
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250687, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.252.keytab.
Entry for principal kudu/127.5.222.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.252.keytab.
I20260809 04:44:48.291790 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.277534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:33989 (local address 127.0.0.1:39214)
0809 04:44:48.277744 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.277762 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:48.277843 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:48.278458 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:48.278463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:48.278676 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:48.278870 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.278883 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.281236 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.281239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.281858 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.281868 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.282530 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.282533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.282607 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.283150 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:48.283169 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:48.286000 (+ 2831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:48.289809 (+ 3809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.289812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.289814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.290058 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.290454 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.290458 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.290461 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.290541 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.291140 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:48.291143 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:48.291226 (+ 83us) client_negotiation.cc:769] Sending connection context
0809 04:44:48.291381 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 04:44:48.291543 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260809 04:44:48.291844 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.277569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:33989 (local address 127.0.0.1:39230)
0809 04:44:48.277737 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.277755 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:48.277843 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:48.278826 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:48.278829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:48.279002 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:48.279110 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.279117 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.280477 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.280484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.281534 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.281552 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.282403 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.282411 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.282573 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.283258 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:48.283284 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:48.286619 (+ 3335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:48.289792 (+ 3173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.289798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.289803 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.290035 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.290409 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.290414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.290417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.290525 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.291211 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:48.291219 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:48.291355 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 04:44:48.291464 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 04:44:48.291633 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 04:44:48.291982 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.252:33785
--webserver_interface=127.5.222.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.252.keytab
--principal=kudu/127.5.222.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:36197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:45207,127.5.222.253:33989,127.5.222.252:33785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:44:48.292097 30926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.277628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39214 (local address 127.5.222.253:33989)
0809 04:44:48.278053 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.278067 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.278094 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:44:48.278140 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.278147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.278309 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.278414 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.279023 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.280308 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.281990 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.282353 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.282438 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.286162 (+ 3724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.286171 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.286181 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.286201 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.289684 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.290180 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.290182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.290184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.290346 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.290645 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.290650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.290655 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.290990 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.291178 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.291491 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.291658 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":180,"server-negotiator.queue_time_us":331,"spinlock_wait_cycles":4352,"thread_start_us":148,"threads_started":1}
I20260809 04:44:48.292640 30925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.277652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39230 (local address 127.5.222.253:33989)
0809 04:44:48.278605 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.278610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.278619 (+ 9us) server_negotiation.cc:407] Connection header received
0809 04:44:48.278641 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.278643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.278668 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.279222 (+ 554us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.279297 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.280308 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.281677 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.282257 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.282359 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.286859 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.286865 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.286868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.286902 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.289649 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.290160 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.290169 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.290171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.290292 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.290645 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.290649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.290651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.291054 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.292342 (+ 1288us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.292413 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.292512 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":203,"server-negotiator.queue_time_us":906,"spinlock_wait_cycles":64128,"thread_start_us":127,"threads_started":1}
I20260809 04:44:48.294292 30920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:48.294571 30840 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:48.295089 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.294817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:38984)
0809 04:44:48.294945 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.295034 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 04:44:48.295343 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.295055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:38990)
0809 04:44:48.295181 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.295244 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 04:44:48.295357 30866 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
I20260809 04:44:48.296248 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.296014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:39008)
0809 04:44:48.296131 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.296194 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 04:44:48.296502 30920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:33785: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
I20260809 04:44:48.297053 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.295913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:38994)
0809 04:44:48.296873 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.296943 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":831,"thread_start_us":636,"threads_started":1}
W20260809 04:44:48.297314 30840 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:33785: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
I20260809 04:44:48.328523 30840 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } attempt: 1
I20260809 04:44:48.329452 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.329140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:39012)
0809 04:44:48.329313 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.329375 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 04:44:48.330235 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.330028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:39026)
0809 04:44:48.330131 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.330184 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 04:44:48.330391 30840 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:33785: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
I20260809 04:44:48.345775 30920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } attempt: 1
I20260809 04:44:48.346621 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.346312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:39030)
0809 04:44:48.346483 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.346544 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 04:44:48.347412 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.347172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:39044)
0809 04:44:48.347304 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.347358 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 04:44:48.347558 30920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:33785: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
W20260809 04:44:48.404436 30927 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:44:48.404742 30927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:48.404814 30927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:48.404904 30927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 04:44:48.405570 30840 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } attempt: 2
I20260809 04:44:48.406488 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.406168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:39052)
0809 04:44:48.406326 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.406393 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 04:44:48.407286 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.407056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:39058)
0809 04:44:48.407176 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.407232 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 04:44:48.407444 30840 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:33785: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
W20260809 04:44:48.409410 30927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:48.409482 30927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:48.409531 30927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:48.409570 30927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:48.409617 30927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:48.414857 30927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:36197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:45207,127.5.222.253:33989,127.5.222.252:33785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.252.keytab
--principal=kudu/127.5.222.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.252:33785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.222.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:48.415755 30920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } attempt: 2
I20260809 04:44:48.415910 30927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:48.416478 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.416182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:39068)
0809 04:44:48.416342 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.416402 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 04:44:48.417208 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.416964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:39072)
0809 04:44:48.417070 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.417121 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 04:44:48.417227 30927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:48.417364 30920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:33785: Network error: Client connection negotiation failed: client connection to 127.5.222.252:33785: connect: Connection refused (error 111)
W20260809 04:44:48.425457 30937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:48.425447 30935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:48.425467 30934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:48.425871 30927 server_base.cc:1034] running on GCE node
I20260809 04:44:48.426417 30927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:48.427511 30927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:48.428711 30927 hybrid_clock.cc:648] HybridClock initialized: now 1786250688428701 us; error 31 us; skew 500 ppm
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:48.431703 30927 init.cc:377] Logged in from keytab as kudu/127.5.222.252@KRBTEST.COM (short username kudu)
I20260809 04:44:48.432559 30927 webserver.cc:460] Webserver started at http://127.5.222.252:38991/ using document root <none> and password file <none>
I20260809 04:44:48.433115 30927 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:48.433233 30927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:48.433496 30927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:48.435549 30927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/data/instance:
uuid: "f0285074eab74c79be1f8c3aae41716c"
format_stamp: "Formatted at 2026-08-09 04:44:48 on dist-test-slave-vxj2"
server_key: "e01498d48136e857a7ecd3c491ab3809"
server_key_iv: "332b7e6c3a5343b9dfde2ac049a2c45a"
server_key_version: "encryptionkey@0"
I20260809 04:44:48.436110 30927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/wal/instance:
uuid: "f0285074eab74c79be1f8c3aae41716c"
format_stamp: "Formatted at 2026-08-09 04:44:48 on dist-test-slave-vxj2"
server_key: "e01498d48136e857a7ecd3c491ab3809"
server_key_iv: "332b7e6c3a5343b9dfde2ac049a2c45a"
server_key_version: "encryptionkey@0"
I20260809 04:44:48.439862 30927 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260809 04:44:48.442296 30944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:48.443492 30927 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:44:48.443631 30927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/wal
uuid: "f0285074eab74c79be1f8c3aae41716c"
format_stamp: "Formatted at 2026-08-09 04:44:48 on dist-test-slave-vxj2"
server_key: "e01498d48136e857a7ecd3c491ab3809"
server_key_iv: "332b7e6c3a5343b9dfde2ac049a2c45a"
server_key_version: "encryptionkey@0"
I20260809 04:44:48.443738 30927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:48.455856 30927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:48.459072 30927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:48.459283 30927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:48.466426 30927 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.252:33785
I20260809 04:44:48.466750 31001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.252:33785 every 8 connection(s)
I20260809 04:44:48.467523 30927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
I20260809 04:44:48.469666 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 30927
I20260809 04:44:48.469792 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250403492198-6011-0/minicluster-data/master-2/wal/instance
I20260809 04:44:48.470103 6011 external_mini_cluster.cc:1428] Setting key ca3eb2feab1cc27d8dc6f9eebb811223
I20260809 04:44:48.470566 31002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:48.475423 31002 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.252@KRBTEST.COM
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.252@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:48.485424 31006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.471387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.5.222.252:33785)
0809 04:44:48.471779 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.471799 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.471834 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:44:48.472475 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.472490 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.472786 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.473079 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.474149 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.475131 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.476426 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.476887 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.477058 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.480100 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.480105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.480114 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.480138 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.482031 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.482634 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.482639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.482643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.482769 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.483159 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.483164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.483166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.483636 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.483879 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.484226 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.484564 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":132,"threads_started":1}
I20260809 04:44:48.487227 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.477686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:45207 (local address 127.0.0.1:53228)
0809 04:44:48.478101 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.478158 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:48.478310 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:48.478620 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:48.478625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:48.479058 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:48.479187 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.479192 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.480091 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.480094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.480809 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.480820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.481431 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.481435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.481529 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.482119 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:48.482139 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:48.483764 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:48.485603 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.485612 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.485618 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.485906 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.486210 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.486213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.486215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.486306 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.486753 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:48.486758 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:48.486856 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 04:44:48.486966 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 04:44:48.487093 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260809 04:44:48.487375 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.477489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53228 (local address 127.5.222.254:45207)
0809 04:44:48.477699 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.477705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.478267 (+ 562us) server_negotiation.cc:407] Connection header received
0809 04:44:48.478428 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.478431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.478476 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.478539 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.479315 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.479930 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.480943 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.481286 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.481418 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.483915 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.483918 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.483921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.483970 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.485477 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.486015 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.486018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.486019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.486108 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.486391 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.486393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.486394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.486642 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.486754 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.487057 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.487190 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:44:48.488837 31002 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.252@KRBTEST.COM for kudu/127.5.222.253@KRBTEST.COM
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:48.497872 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.489365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:33989 (local address 127.0.0.1:39246)
0809 04:44:48.489467 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.489481 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:48.489553 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:48.489880 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:48.489882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:48.490056 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:48.490159 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.490165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.491140 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.491142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.491743 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.491752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.492355 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.492360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.492476 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.492932 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:48.492943 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:48.494205 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:48.496207 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.496211 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.496213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.496501 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.496799 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.496802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.496804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.496889 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.497369 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:48.497372 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:48.497466 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 04:44:48.497570 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 04:44:48.497697 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 04:44:48.497885 30925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.489496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39246 (local address 127.5.222.253:33989)
0809 04:44:48.489643 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.489649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.489664 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:44:48.489709 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.489713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.489759 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.489853 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.490299 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.491026 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.491858 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.492205 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.492345 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.494353 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.494357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.494360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.494392 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.496088 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.496604 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.496606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.496607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.496694 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.496987 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.496991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.496992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.497225 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.497378 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.497623 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.497739 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:44:48.498647 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.488388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.5.222.254:45207)
0809 04:44:48.488520 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.488525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.488598 (+ 73us) server_negotiation.cc:407] Connection header received
0809 04:44:48.488759 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.488761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.488801 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.488864 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.489958 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.490562 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.491554 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.491918 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.491993 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.494942 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.494946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.494948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.494969 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.496704 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.497186 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.497190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.497192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.497284 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.497586 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.497591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.497592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.497821 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.498084 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.498312 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.498441 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:44:48.499079 31002 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.252@KRBTEST.COM for kudu/127.5.222.252@KRBTEST.COM
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.253@KRBTEST.COM
I20260809 04:44:48.506613 30840 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } attempt: 3
I20260809 04:44:48.507907 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.499612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:39086)
0809 04:44:48.499742 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.499755 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:48.499848 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:48.500077 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:48.500080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:48.500232 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:48.500295 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.500299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.500979 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.500984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.501579 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.501588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.502475 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.502479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.502588 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.503040 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:48.503055 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:48.504300 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:48.506261 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.506265 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.506268 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.506526 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.506827 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.506829 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.506831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.506918 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.507360 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:48.507363 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:48.507452 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 04:44:48.507568 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 04:44:48.507723 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":22}
I20260809 04:44:48.508018 31006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.499653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.5.222.252:33785)
0809 04:44:48.499754 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.499758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.499808 (+ 50us) server_negotiation.cc:407] Connection header received
0809 04:44:48.499944 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.499949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.499982 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.500067 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.500398 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.500882 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.501689 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.502369 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.502479 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.504442 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.504446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.504448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.504470 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.506141 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.506624 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.506631 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.506636 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.506721 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.507024 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.507027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.507028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.507212 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.507309 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.507627 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.507811 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":36}
I20260809 04:44:48.508606 30920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } attempt: 3
I20260809 04:44:48.509333 30925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.500028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39250 (local address 127.5.222.253:33989)
0809 04:44:48.500146 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.500151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.500171 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:44:48.500200 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.500202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.500235 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.500328 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.500951 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.501555 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.502370 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.502608 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.502719 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.505435 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.505439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.505442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.505470 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.507554 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.508079 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.508082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.508084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.508159 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.508497 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.508506 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.508507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.508663 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.508914 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.509023 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.509131 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250687, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.252@KRBTEST.COM
I20260809 04:44:48.513406 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:48.513652 31002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c: Bootstrap starting.
Aug 09 04:44:48 dist-test-slave-vxj2 krb5kdc[30751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.253@KRBTEST.COM for kudu/127.5.222.252@KRBTEST.COM
I20260809 04:44:48.514894 30810 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "277351e526d74a21b7473618b736048f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53240
I20260809 04:44:48.516685 31002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:48.517617 31002 log.cc:824] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:48.518230 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.507093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:39088)
0809 04:44:48.507195 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.507210 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:48.507299 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:48.507903 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:48.507906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:48.508124 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:48.508285 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.508293 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.509389 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.509392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.510246 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.510256 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.511135 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.511139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.511264 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.511888 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:48.511909 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:48.513529 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:48.516409 (+ 2880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.516413 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.516417 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.516699 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.517060 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.517064 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.517066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.517210 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.517659 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:48.517662 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:48.517774 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 04:44:48.517894 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 04:44:48.518055 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 04:44:48.518903 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.507270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39088 (local address 127.5.222.252:33785)
0809 04:44:48.507614 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.507623 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.507635 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:44:48.507664 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.507666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.507705 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.508406 (+ 701us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.508507 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.509261 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.510631 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.511002 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.511105 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.513672 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.513679 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.513684 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.513710 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.516272 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.516858 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.516862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.516865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.516963 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.517318 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.517323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.517326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.517545 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.517683 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.518617 (+ 934us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.518735 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":151,"threads_started":1}
I20260809 04:44:48.519650 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.508937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:33785 (local address 127.0.0.1:39090)
0809 04:44:48.509192 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.509205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:48.509273 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:48.510040 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:48.510043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:48.510246 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:48.510345 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.510352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.511939 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.511942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.512765 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.512779 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.513493 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.513497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.513594 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.514269 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:48.514280 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:48.515787 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:48.517933 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.517936 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.517938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.518191 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.518516 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:48.518520 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:48.518522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:48.518615 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:48.519126 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:48.519130 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:48.519240 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 04:44:48.519355 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 04:44:48.519492 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260809 04:44:48.519676 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.509429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39090 (local address 127.5.222.252:33785)
0809 04:44:48.509790 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.509798 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.509808 (+ 10us) server_negotiation.cc:407] Connection header received
0809 04:44:48.509848 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.509850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.509906 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.510447 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.510550 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.511788 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.512942 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.513342 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.514798 (+ 1456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.515944 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.515948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.515951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.515979 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.517818 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.518311 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.518314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.518316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.518398 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.518718 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.518724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.518726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.518968 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.519080 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.519417 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.519525 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":154,"threads_started":1}
I20260809 04:44:48.520407 31002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c: No bootstrap required, opened a new log
I20260809 04:44:48.522164 30840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f: Bootstrap starting.
I20260809 04:44:48.522622 30920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342: Bootstrap starting.
I20260809 04:44:48.524430 31002 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } }
I20260809 04:44:48.524668 31002 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0285074eab74c79be1f8c3aae41716c, State: Initialized, Role: FOLLOWER
I20260809 04:44:48.524789 30840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:48.525074 30920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:48.525125 31002 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } }
I20260809 04:44:48.525797 30840 log.cc:824] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:48.525995 30920 log.cc:824] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:48.525877 31011 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } } }
I20260809 04:44:48.526258 31011 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:48.527369 31002 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:48.528402 30920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342: No bootstrap required, opened a new log
I20260809 04:44:48.528720 30840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f: No bootstrap required, opened a new log
I20260809 04:44:48.531831 30840 raft_consensus.cc:348] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } }
I20260809 04:44:48.532064 30840 raft_consensus.cc:729] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277351e526d74a21b7473618b736048f, State: Initialized, Role: FOLLOWER
I20260809 04:44:48.532378 30840 consensus_queue.cc:260] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } }
I20260809 04:44:48.532536 30920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } }
I20260809 04:44:48.532732 30920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 773f4d4e93484a70b4941054e9c4b342, State: Initialized, Role: FOLLOWER
W20260809 04:44:48.533010 31025 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:48.533087 31025 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:48.533011 30920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } }
I20260809 04:44:48.533684 30840 sys_catalog.cc:564] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:48.534160 30920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:48.531368 30927 master_runner.cc:186] Error getting master registration for 127.5.222.254:45207: OK, instance_id { permanent_uuid: "277351e526d74a21b7473618b736048f" instance_seqno: 1786250688032664 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:44:48.534322 30927 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:44:48.534549 30810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 04:44:48.534665 30810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:48.535858 30810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } }
I20260809 04:44:48.536754 30810 leader_election.cc:288] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [CANDIDATE]: Term 1 election: Requested vote from peers 773f4d4e93484a70b4941054e9c4b342 (127.5.222.253:33989), f0285074eab74c79be1f8c3aae41716c (127.5.222.252:33785)
I20260809 04:44:48.537325 30892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "277351e526d74a21b7473618b736048f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "773f4d4e93484a70b4941054e9c4b342"
I20260809 04:44:48.537535 30892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:48.537592 30973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "277351e526d74a21b7473618b736048f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f0285074eab74c79be1f8c3aae41716c"
I20260809 04:44:48.537791 30973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:48.538834 30892 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 277351e526d74a21b7473618b736048f in term 1.
I20260809 04:44:48.539109 30973 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 277351e526d74a21b7473618b736048f in term 1.
I20260809 04:44:48.539501 30782 leader_election.cc:302] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 277351e526d74a21b7473618b736048f, f0285074eab74c79be1f8c3aae41716c; no voters:
W20260809 04:44:48.541464 31045 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:48.541606 31045 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:48.539273 30764 master_runner.cc:186] Error getting master registration for 127.5.222.253:33989: OK, instance_id { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" instance_seqno: 1786250688250852 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:44:48.542757 30764 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 04:44:48.545102 31059 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:48.545266 31059 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:48.548516 30847 master_runner.cc:263] No leader master found from master 773f4d4e93484a70b4941054e9c4b342
I20260809 04:44:48.548589 30847 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:44:48.553484 31027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } } }
I20260809 04:44:48.553659 31027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:48.553792 31027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:48.553905 31027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [term 1 LEADER]: Becoming Leader. State: Replica: 277351e526d74a21b7473618b736048f, State: Running, Role: LEADER
I20260809 04:44:48.554288 31027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } }
I20260809 04:44:48.556408 31027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 277351e526d74a21b7473618b736048f. Latest consensus state: current_term: 1 leader_uuid: "277351e526d74a21b7473618b736048f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } } }
I20260809 04:44:48.556540 31027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:48.556845 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.541376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53254 (local address 127.5.222.254:45207)
0809 04:44:48.542913 (+ 1537us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.542919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.546978 (+ 4059us) server_negotiation.cc:407] Connection header received
0809 04:44:48.547218 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.547222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.547274 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.547342 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.548561 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.549443 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.550711 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.551052 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.551130 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.553090 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.553095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.553098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.553125 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.554907 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.555429 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.555433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.555435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.555520 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.555837 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.555840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.555842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.556047 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.556150 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.556558 (+ 408us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.556687 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1461}
I20260809 04:44:48.564577 31062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:48.565824 31062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:48.567770 30925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.541998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.5.222.253:33989)
0809 04:44:48.542108 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.542112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.557538 (+ 15426us) server_negotiation.cc:407] Connection header received
0809 04:44:48.557719 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.557723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.557771 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.557824 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.558870 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.559663 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.560742 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.561070 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.561140 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.562889 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.562893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.562896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.562922 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.564888 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.565422 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.565425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.565427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.565519 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.565843 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.565849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.565850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.566070 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.566190 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.567480 (+ 1290us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.567605 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:44:48.568302 31028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } } }
I20260809 04:44:48.568460 31028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:48.573257 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.557052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39094 (local address 127.5.222.252:33785)
0809 04:44:48.557196 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.557201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.558405 (+ 1204us) server_negotiation.cc:407] Connection header received
0809 04:44:48.558594 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.558598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.558650 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.558702 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.559597 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.560419 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.563282 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.563703 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.563806 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.569416 (+ 5610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:48.569420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:48.569423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:48.569453 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:48.571283 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.571804 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.571808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.571810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.571904 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:48.572222 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:48.572225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:48.572227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:48.572449 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:48.572576 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:48.572941 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 04:44:48.573070 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:44:48.578358 30973 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [term 1 FOLLOWER]: Refusing update from remote peer 277351e526d74a21b7473618b736048f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:48.579147 31051 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:48.583673 31011 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 277351e526d74a21b7473618b736048f. Latest consensus state: current_term: 1 leader_uuid: "277351e526d74a21b7473618b736048f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } } }
I20260809 04:44:48.583808 31011 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:48.584796 30892 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [term 1 FOLLOWER]: Refusing update from remote peer 277351e526d74a21b7473618b736048f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:48.585675 31027 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [LEADER]: Connected to new peer: Peer: permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:48.589581 31028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 277351e526d74a21b7473618b736048f. Latest consensus state: current_term: 1 leader_uuid: "277351e526d74a21b7473618b736048f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } } }
I20260809 04:44:48.589694 31028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:48.594039 31051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "277351e526d74a21b7473618b736048f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } } }
I20260809 04:44:48.594163 31051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:48.595391 31028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "277351e526d74a21b7473618b736048f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } } }
I20260809 04:44:48.595494 31028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:48.595948 31063 mvcc.cc:205] Tried to move back new op lower bound from 7316482820403728384 to 7316482820322693120. Current Snapshot: MvccSnapshot[applied={T|T < 7316482820403728384}]
I20260809 04:44:48.596344 31066 mvcc.cc:205] Tried to move back new op lower bound from 7316482820403728384 to 7316482820322693120. Current Snapshot: MvccSnapshot[applied={T|T < 7316482820403728384}]
I20260809 04:44:48.598476 31027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "277351e526d74a21b7473618b736048f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } } }
I20260809 04:44:48.598604 31027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:48.599604 31011 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "277351e526d74a21b7473618b736048f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277351e526d74a21b7473618b736048f" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 45207 } } peers { permanent_uuid: "773f4d4e93484a70b4941054e9c4b342" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 33989 } } peers { permanent_uuid: "f0285074eab74c79be1f8c3aae41716c" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 33785 } } }
I20260809 04:44:48.599720 31011 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:48.602316 31064 mvcc.cc:205] Tried to move back new op lower bound from 7316482820403728384 to 7316482820322693120. Current Snapshot: MvccSnapshot[applied={T|T < 7316482820403728384}]
I20260809 04:44:48.605492 31062 catalog_manager.cc:1347] Generated new cluster ID: 825fd98c1db04dbc8631d47c431515d5
I20260809 04:44:48.605568 31062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:48.623745 31062 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:48.625000 31062 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:48.642658 31062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 277351e526d74a21b7473618b736048f: Generated new TSK 0
I20260809 04:44:48.643558 31062 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:44:48.676678 31009 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.673635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39106 (local address 127.5.222.252:33785)
0809 04:44:48.673759 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.673764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.675562 (+ 1798us) server_negotiation.cc:407] Connection header received
0809 04:44:48.675770 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.675775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.675825 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.675879 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.676543 (+ 664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39106: BlockingRecv error: recv got EOF from 127.0.0.1:39106 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 04:44:48.678108 30843 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.674371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53264 (local address 127.5.222.254:45207)
0809 04:44:48.674485 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.674490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.677217 (+ 2727us) server_negotiation.cc:407] Connection header received
0809 04:44:48.677398 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.677402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.677453 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.677505 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.677993 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53264: BlockingRecv error: recv got EOF from 127.0.0.1:53264 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 04:44:48.679425 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.674966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39260 (local address 127.5.222.253:33989)
0809 04:44:48.675074 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.675078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.678576 (+ 3498us) server_negotiation.cc:407] Connection header received
0809 04:44:48.678747 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.678751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.678799 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.678848 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.679304 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39260: BlockingRecv error: recv got EOF from 127.0.0.1:39260 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 04:44:48.683295 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.681825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39266 (local address 127.5.222.253:33989)
0809 04:44:48.681941 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.681945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.682417 (+ 472us) server_negotiation.cc:407] Connection header received
0809 04:44:48.682617 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.682620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.682687 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.682747 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.683200 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39266: BlockingRecv error: recv got EOF from 127.0.0.1:39266 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 04:44:48.688073 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.686760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39270 (local address 127.5.222.253:33989)
0809 04:44:48.686853 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.686857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.687269 (+ 412us) server_negotiation.cc:407] Connection header received
0809 04:44:48.687435 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.687438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.687481 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.687528 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.687987 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39270: BlockingRecv error: recv got EOF from 127.0.0.1:39270 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 04:44:48.693639 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.692121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39272 (local address 127.5.222.253:33989)
0809 04:44:48.692221 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.692227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.692683 (+ 456us) server_negotiation.cc:407] Connection header received
0809 04:44:48.692862 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.692865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.692910 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.692968 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.693549 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39272: BlockingRecv error: recv got EOF from 127.0.0.1:39272 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 04:44:48.701455 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.699833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39286 (local address 127.5.222.253:33989)
0809 04:44:48.699966 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.699970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.700499 (+ 529us) server_negotiation.cc:407] Connection header received
0809 04:44:48.700690 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.700694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.700754 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.700807 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.701359 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39286: BlockingRecv error: recv got EOF from 127.0.0.1:39286 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 04:44:48.713563 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.711864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39300 (local address 127.5.222.253:33989)
0809 04:44:48.711979 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.711983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.712529 (+ 546us) server_negotiation.cc:407] Connection header received
0809 04:44:48.712724 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.712727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.712786 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.712845 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.713453 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39300: BlockingRecv error: recv got EOF from 127.0.0.1:39300 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 04:44:48.734472 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.732581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39316 (local address 127.5.222.253:33989)
0809 04:44:48.732709 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.732713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.733351 (+ 638us) server_negotiation.cc:407] Connection header received
0809 04:44:48.733570 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.733574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.733644 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.733698 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.734356 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39316: BlockingRecv error: recv got EOF from 127.0.0.1:39316 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 04:44:48.770571 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.769246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39318 (local address 127.5.222.253:33989)
0809 04:44:48.769381 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.769385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.769657 (+ 272us) server_negotiation.cc:407] Connection header received
0809 04:44:48.769761 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.769766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.769849 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.769969 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.770432 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39318: BlockingRecv error: recv got EOF from 127.0.0.1:39318 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 04:44:48.838034 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.836825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39320 (local address 127.5.222.253:33989)
0809 04:44:48.837041 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.837045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.837103 (+ 58us) server_negotiation.cc:407] Connection header received
0809 04:44:48.837284 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.837286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.837336 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.837412 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.837859 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39320: BlockingRecv error: recv got EOF from 127.0.0.1:39320 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260809 04:44:48.969383 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.968155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39334 (local address 127.5.222.253:33989)
0809 04:44:48.968305 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.968308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.968485 (+ 177us) server_negotiation.cc:407] Connection header received
0809 04:44:48.968607 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.968609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.968657 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.968742 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.969285 (+ 543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39334: BlockingRecv error: recv got EOF from 127.0.0.1:39334 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 04:44:49.229030 30925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:49.227825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39346 (local address 127.5.222.253:33989)
0809 04:44:49.227966 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:49.227981 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:49.228143 (+ 162us) server_negotiation.cc:407] Connection header received
0809 04:44:49.228264 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:49.228266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:49.228314 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:49.228386 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:49.228882 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39346: BlockingRecv error: recv got EOF from 127.0.0.1:39346 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 04:44:49.534700 31025 catalog_manager.cc:1259] Loaded cluster ID: 825fd98c1db04dbc8631d47c431515d5
I20260809 04:44:49.534801 31025 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c: loading cluster ID for follower catalog manager: success
I20260809 04:44:49.536733 31025 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c: acquiring CA information for follower catalog manager: success
I20260809 04:44:49.537917 31025 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f0285074eab74c79be1f8c3aae41716c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:44:49.546525 31059 catalog_manager.cc:1259] Loaded cluster ID: 825fd98c1db04dbc8631d47c431515d5
I20260809 04:44:49.546592 31059 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342: loading cluster ID for follower catalog manager: success
I20260809 04:44:49.548386 31059 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342: acquiring CA information for follower catalog manager: success
I20260809 04:44:49.549542 31059 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 773f4d4e93484a70b4941054e9c4b342: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:44:49.747011 31141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:49.743130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39350 (local address 127.5.222.253:33989)
0809 04:44:49.743372 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:49.743376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:49.743391 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:44:49.743537 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:49.743539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:49.743661 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:49.743808 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:49.744065 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.744590 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.745473 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.745706 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.745843 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:49.746110 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:49.746559 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:49.746657 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:49.746710 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 04:44:49.746832 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":70,"threads_started":1}
I20260809 04:44:49.754086 31147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:49.750356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39112 (local address 127.5.222.252:33785)
0809 04:44:49.750611 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:49.750616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:49.750630 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:44:49.750793 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:49.750796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:49.750916 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:49.751044 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:49.751258 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.751783 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.752605 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.752843 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.752956 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:49.753272 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:49.753688 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:49.753826 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:49.753867 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 04:44:49.753923 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":79,"threads_started":1}
I20260809 04:44:49.761495 31154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:49.757042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.5.222.254:45207)
0809 04:44:49.757344 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:49.757450 (+ 106us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:49.757465 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:44:49.757569 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:49.757572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:49.757750 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:49.757906 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:49.758136 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.758828 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.759894 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.760118 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.760232 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:49.760555 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:49.760884 (+ 329us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:49.760970 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:49.761268 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 04:44:49.761341 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":105,"threads_started":1}
I20260809 04:44:49.761549 31141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:49.757327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39362 (local address 127.5.222.253:33989)
0809 04:44:49.757426 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:49.757431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:49.757766 (+ 335us) server_negotiation.cc:407] Connection header received
0809 04:44:49.758066 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:49.758069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:49.758122 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:49.758189 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:49.758550 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.759174 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.760349 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.760629 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.760735 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:49.761057 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:49.761133 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:49.761226 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:49.761359 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 04:44:49.761416 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 04:44:49.762149 31147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:49.757134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.5.222.252:33785)
0809 04:44:49.757267 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:49.757271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:49.757718 (+ 447us) server_negotiation.cc:407] Connection header received
0809 04:44:49.757847 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:49.757853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:49.757906 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:49.757977 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:49.758190 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.758820 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.760871 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.761188 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.761296 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:49.761668 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:49.761753 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:49.761804 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:49.761979 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 04:44:49.762032 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:44:49.765070 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 30764
I20260809 04:44:49.774194 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 30847
I20260809 04:44:49.780880 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 30927
2026-08-09T04:44:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3166 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:49 dist-test-slave-vxj2 krb5kdc[31159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:49 dist-test-slave-vxj2 krb5kdc[31159](info): set up 2 sockets
Aug 09 04:44:49 dist-test-slave-vxj2 krb5kdc[31159](info): commencing operation
krb5kdc: starting...
W20260809 04:44:50.913995 6011 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/test-user.keytab.
Aug 09 04:44:50 dist-test-slave-vxj2 krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.222.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
Entry for principal kudu/127.5.222.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab.
I20260809 04:44:51.039644 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:33909
--webserver_interface=127.5.222.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:33909,127.5.222.253:45811,127.5.222.252:42997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:51.149757 31172 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:44:51.150038 31172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:51.150060 31172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:51.150137 31172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:51.154103 31172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:51.154162 31172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:51.154181 31172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:51.154198 31172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:51.154212 31172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:51.159391 31172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:33909,127.5.222.253:45811,127.5.222.252:42997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.254.keytab
--principal=kudu/127.5.222.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.254:33909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.222.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:51.160440 31172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:51.161520 31172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:51.170418 31178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:51.170444 31177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:51.170511 31172 server_base.cc:1034] running on GCE node
W20260809 04:44:51.170418 31180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:51.171236 31172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:51.172282 31172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:51.173439 31172 hybrid_clock.cc:648] HybridClock initialized: now 1786250691173423 us; error 29 us; skew 500 ppm
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250691, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:51.176182 31172 init.cc:377] Logged in from keytab as kudu/127.5.222.254@KRBTEST.COM (short username kudu)
I20260809 04:44:51.177008 31172 webserver.cc:460] Webserver started at http://127.5.222.254:46141/ using document root <none> and password file <none>
I20260809 04:44:51.177572 31172 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:51.177635 31172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:51.177891 31172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:51.180055 31172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/data/instance:
uuid: "1d69441040a1479a9f96339d35726d80"
format_stamp: "Formatted at 2026-08-09 04:44:51 on dist-test-slave-vxj2"
server_key: "d14ab1d4142d86b1beeafe54dc2ef37e"
server_key_iv: "541a6ed66fe9038fcc74c2a2615334dd"
server_key_version: "encryptionkey@0"
I20260809 04:44:51.180627 31172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/wal/instance:
uuid: "1d69441040a1479a9f96339d35726d80"
format_stamp: "Formatted at 2026-08-09 04:44:51 on dist-test-slave-vxj2"
server_key: "d14ab1d4142d86b1beeafe54dc2ef37e"
server_key_iv: "541a6ed66fe9038fcc74c2a2615334dd"
server_key_version: "encryptionkey@0"
I20260809 04:44:51.184405 31172 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 04:44:51.186749 31187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:51.187920 31172 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:44:51.188056 31172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/wal
uuid: "1d69441040a1479a9f96339d35726d80"
format_stamp: "Formatted at 2026-08-09 04:44:51 on dist-test-slave-vxj2"
server_key: "d14ab1d4142d86b1beeafe54dc2ef37e"
server_key_iv: "541a6ed66fe9038fcc74c2a2615334dd"
server_key_version: "encryptionkey@0"
I20260809 04:44:51.188164 31172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:51.229408 31172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:51.232376 31172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:51.232595 31172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:51.239662 31172 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.254:33909
I20260809 04:44:51.239979 31245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.254:33909 every 8 connection(s)
I20260809 04:44:51.240674 31172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:51.243429 31246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:51.246169 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 31172
I20260809 04:44:51.246335 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-0/wal/instance
I20260809 04:44:51.246636 6011 external_mini_cluster.cc:1428] Setting key fb609bfe3e07ac9b94c0d47ef604d954
I20260809 04:44:51.247278 31246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.222.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.253@KRBTEST.COM" created.
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250691, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:51.264178 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.249674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:33909 (local address 127.0.0.1:44542)
0809 04:44:51.250208 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.250268 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:51.251462 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:51.252170 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:51.252179 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:51.252703 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:51.253100 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.253124 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.254317 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.254321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.255622 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.255633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.256509 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.256512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.256590 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.257191 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:51.257217 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:51.259396 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:51.261432 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.261443 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.261453 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.261668 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.262004 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.262007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.262008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.262125 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.262955 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:51.262960 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:51.263074 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 04:44:51.263201 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 04:44:51.263440 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":126,"threads_started":1}
I20260809 04:44:51.264181 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.250373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.5.222.254:33909)
0809 04:44:51.250628 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.250637 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.250672 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:44:51.251762 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.251766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.251953 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.252103 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.253287 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.254176 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.255885 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.256263 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.256533 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.259545 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.259550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.259559 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.259580 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.261332 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.261791 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.261796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.261797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.261891 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.262279 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.262285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.262290 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.262831 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.262994 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.263355 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.263528 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":91,"threads_started":1}
I20260809 04:44:51.266609 31246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:51.267354 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.267086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:45811 (local address 127.0.0.1:50078)
0809 04:44:51.267199 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.267266 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 04:44:51.267619 31188 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.222.253:45811: connect: Connection refused (error 111)
I20260809 04:44:51.268347 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.268178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:45811 (local address 127.0.0.1:50090)
0809 04:44:51.268258 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.268300 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 04:44:51.268554 31246 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.253:45811: Network error: Client connection negotiation failed: client connection to 127.5.222.253:45811: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.253.keytab.
Entry for principal kudu/127.5.222.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.253.keytab.
I20260809 04:44:51.276368 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.253:45811
--webserver_interface=127.5.222.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.253.keytab
--principal=kudu/127.5.222.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:33909,127.5.222.253:45811,127.5.222.252:42997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:44:51.317947 31246 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } attempt: 1
I20260809 04:44:51.318914 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.318601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:45811 (local address 127.0.0.1:50106)
0809 04:44:51.318770 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.318835 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 04:44:51.319656 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.319456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:45811 (local address 127.0.0.1:50122)
0809 04:44:51.319556 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.319601 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 04:44:51.319808 31246 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.253:45811: Network error: Client connection negotiation failed: client connection to 127.5.222.253:45811: connect: Connection refused (error 111)
I20260809 04:44:51.388211 31246 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } attempt: 2
I20260809 04:44:51.389204 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.388808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:45811 (local address 127.0.0.1:50124)
0809 04:44:51.388993 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.389077 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 04:44:51.390057 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.389810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:45811 (local address 127.0.0.1:50138)
0809 04:44:51.389907 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.389985 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.253:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 04:44:51.390239 31246 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.253:45811: Network error: Client connection negotiation failed: client connection to 127.5.222.253:45811: connect: Connection refused (error 111)
W20260809 04:44:51.395126 31254 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:44:51.395435 31254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:51.395498 31254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:51.395578 31254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:51.399668 31254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:51.399750 31254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:51.399797 31254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:51.399854 31254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:51.399887 31254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:51.405499 31254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:33909,127.5.222.253:45811,127.5.222.252:42997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.253.keytab
--principal=kudu/127.5.222.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.253:45811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.222.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:51.406559 31254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:51.407562 31254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:44:51.416373 31254 server_base.cc:1034] running on GCE node
W20260809 04:44:51.416311 31259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:51.416306 31262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:51.416306 31260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:51.417074 31254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:51.418141 31254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:51.419303 31254 hybrid_clock.cc:648] HybridClock initialized: now 1786250691419289 us; error 27 us; skew 500 ppm
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250691, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:51.422097 31254 init.cc:377] Logged in from keytab as kudu/127.5.222.253@KRBTEST.COM (short username kudu)
I20260809 04:44:51.423115 31254 webserver.cc:460] Webserver started at http://127.5.222.253:41703/ using document root <none> and password file <none>
I20260809 04:44:51.423714 31254 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:51.423791 31254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:51.424048 31254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:51.426332 31254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/data/instance:
uuid: "f60f522ce9264954ae783c735ddbb83c"
format_stamp: "Formatted at 2026-08-09 04:44:51 on dist-test-slave-vxj2"
server_key: "3f0888d9e0c71acf9e7c0e2174be7c65"
server_key_iv: "e4c362568b41e288322f3f8dc4ca8793"
server_key_version: "encryptionkey@0"
I20260809 04:44:51.426990 31254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/wal/instance:
uuid: "f60f522ce9264954ae783c735ddbb83c"
format_stamp: "Formatted at 2026-08-09 04:44:51 on dist-test-slave-vxj2"
server_key: "3f0888d9e0c71acf9e7c0e2174be7c65"
server_key_iv: "e4c362568b41e288322f3f8dc4ca8793"
server_key_version: "encryptionkey@0"
I20260809 04:44:51.430924 31254 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:51.433450 31270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:51.434662 31254 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:44:51.434800 31254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/wal
uuid: "f60f522ce9264954ae783c735ddbb83c"
format_stamp: "Formatted at 2026-08-09 04:44:51 on dist-test-slave-vxj2"
server_key: "3f0888d9e0c71acf9e7c0e2174be7c65"
server_key_iv: "e4c362568b41e288322f3f8dc4ca8793"
server_key_version: "encryptionkey@0"
I20260809 04:44:51.434911 31254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:51.455694 31254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:51.458868 31254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:51.459079 31254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:51.465844 31254 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.253:45811
I20260809 04:44:51.466193 31328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.253:45811 every 8 connection(s)
I20260809 04:44:51.466823 31254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/data/info.pb
I20260809 04:44:51.469869 31329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:51.473183 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 31254
I20260809 04:44:51.473351 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-1/wal/instance
I20260809 04:44:51.473647 6011 external_mini_cluster.cc:1428] Setting key 1522a2f3caed30e5b456240b5e94564f
I20260809 04:44:51.474385 31329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:51.481376 31246 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } attempt: 3
WARNING: no policy specified for kudu/127.5.222.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.222.252@KRBTEST.COM" created.
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250691, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.253@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250691, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.253@KRBTEST.COM
I20260809 04:44:51.492404 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.481871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:45811 (local address 127.0.0.1:50146)
0809 04:44:51.482065 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.482082 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:51.482161 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:51.482712 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:51.482714 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:51.482913 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:51.483069 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.483078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.484211 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.484214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.484771 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.484780 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.485467 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.485470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.485605 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.486200 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:51.486218 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:51.488081 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:51.490615 (+ 2534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.490620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.490622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.490827 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.491283 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.491290 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.491295 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.491388 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.491877 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:51.491881 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:51.491994 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 04:44:51.492128 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 04:44:51.492274 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 04:44:51.493438 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.481942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.5.222.253:45811)
0809 04:44:51.482361 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.482368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.482387 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:44:51.482420 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.482423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.482558 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.482735 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.483217 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.484095 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.484890 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.485354 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.485452 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.488261 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.488268 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.488282 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.488311 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.490475 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.490994 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.491000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.491002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.491150 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.491504 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.491506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.491507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.491758 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.491908 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.492683 (+ 775us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.492853 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":97,"threads_started":1}
I20260809 04:44:51.493449 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.476811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:33909 (local address 127.0.0.1:44554)
0809 04:44:51.477310 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.477354 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:51.478170 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:51.478755 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:51.478761 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:51.479098 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:51.479546 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.479558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.480497 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.480500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.481422 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.481432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.482448 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.482456 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.482688 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.484112 (+ 1424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:51.484137 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:51.486795 (+ 2658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:51.489729 (+ 2934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.489742 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.489754 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.490265 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.490632 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.490638 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.490641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.490839 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.491391 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:51.491401 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:51.491764 (+ 363us) client_negotiation.cc:769] Sending connection context
0809 04:44:51.491959 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 04:44:51.492365 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":106,"threads_started":1}
I20260809 04:44:51.494088 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.476490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.5.222.254:33909)
0809 04:44:51.476635 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.476638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.477457 (+ 819us) server_negotiation.cc:407] Connection header received
0809 04:44:51.478414 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.478418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.478480 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.478593 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.479706 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.480357 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.481684 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.482103 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.482237 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.486931 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.486943 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.486946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.486978 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.489598 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.490407 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.490412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.490417 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.490518 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.491006 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.491010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.491012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.491266 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.491370 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.493386 (+ 2016us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.493517 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:44:51.495229 31329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:51.495999 31246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:51.496727 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.496482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34836)
0809 04:44:51.496624 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.496678 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 04:44:51.497522 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.497272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34840)
0809 04:44:51.497429 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.497476 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260809 04:44:51.497681 31246 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:42997: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250691, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.253@KRBTEST.COM for kudu/127.5.222.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.222.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.252.keytab.
Entry for principal kudu/127.5.222.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.252.keytab.
I20260809 04:44:51.502420 6011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
/tmp/dist-test-taskxJEoz3/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.252:42997
--webserver_interface=127.5.222.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.252.keytab
--principal=kudu/127.5.222.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.222.212:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.222.254:33909,127.5.222.253:45811,127.5.222.252:42997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:44:51.504186 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.495794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50160 (local address 127.5.222.253:45811)
0809 04:44:51.496050 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.496055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.496079 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:44:51.496118 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.496121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.496166 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.496726 (+ 560us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.496778 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.497512 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.498356 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.498627 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.498768 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.500593 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.500597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.500600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.500624 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.502225 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.502760 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.502766 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.502771 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.502864 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.503198 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.503204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.503209 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.503456 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.503564 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.503885 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.504055 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":176}
I20260809 04:44:51.504236 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.495735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:45811 (local address 127.0.0.1:50160)
0809 04:44:51.495925 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.495937 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:51.495997 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:51.496295 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:51.496301 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:51.496517 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:51.496626 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.496633 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.497626 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.497632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.498223 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.498232 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.498767 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.498771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.498867 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.499250 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:51.499262 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:51.500473 (+ 1211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:51.502341 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.502347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.502351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.502631 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.503002 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.503005 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.503006 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.503094 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.503581 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:51.503583 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:51.503692 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 04:44:51.503824 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 04:44:51.503984 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260809 04:44:51.505007 31329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:51.505777 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.505459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34852)
0809 04:44:51.505611 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.505700 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 04:44:51.506009 31274 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
I20260809 04:44:51.506927 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.506643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34858)
0809 04:44:51.506789 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.506851 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 04:44:51.507100 31329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:42997: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
I20260809 04:44:51.528951 31246 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } attempt: 1
I20260809 04:44:51.529879 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.529516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34860)
0809 04:44:51.529711 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.529785 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 04:44:51.530637 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.530408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34874)
0809 04:44:51.530521 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.530574 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 04:44:51.530782 31246 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:42997: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
I20260809 04:44:51.556396 31329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } attempt: 1
I20260809 04:44:51.557345 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.556979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34882)
0809 04:44:51.557144 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.557245 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 04:44:51.558647 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.558400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34888)
0809 04:44:51.558522 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.558576 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:44:51.558823 31329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:42997: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
I20260809 04:44:51.605955 31246 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } attempt: 2
I20260809 04:44:51.606845 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.606541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34896)
0809 04:44:51.606699 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.606763 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 04:44:51.607622 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.607385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34906)
0809 04:44:51.607512 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.607562 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 04:44:51.607761 31246 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:42997: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
W20260809 04:44:51.614887 31335 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:44:51.615134 31335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:51.615207 31335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:51.615312 31335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:51.619359 31335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:51.619413 31335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:51.619433 31335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:51.619489 31335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:51.619524 31335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:51.624819 31335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.222.212:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.222.254:33909,127.5.222.253:45811,127.5.222.252:42997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/krb5kdc/kudu_127.5.222.252.keytab
--principal=kudu/127.5.222.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.222.252:42997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.222.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:51.626019 31335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:51.627081 31335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:44:51.627027 31329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } attempt: 2
I20260809 04:44:51.627856 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.627521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34912)
0809 04:44:51.627695 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.627769 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 04:44:51.628674 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.628386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34924)
0809 04:44:51.628535 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.628614 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:44:51.628835 31329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.222.252:42997: Network error: Client connection negotiation failed: client connection to 127.5.222.252:42997: connect: Connection refused (error 111)
W20260809 04:44:51.635785 31342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:51.635782 31344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:51.635824 31341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:51.636262 31335 server_base.cc:1034] running on GCE node
I20260809 04:44:51.636742 31335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:51.637843 31335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:51.638984 31335 hybrid_clock.cc:648] HybridClock initialized: now 1786250691638962 us; error 31 us; skew 500 ppm
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250691, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:51.641712 31335 init.cc:377] Logged in from keytab as kudu/127.5.222.252@KRBTEST.COM (short username kudu)
I20260809 04:44:51.642573 31335 webserver.cc:460] Webserver started at http://127.5.222.252:46583/ using document root <none> and password file <none>
I20260809 04:44:51.643103 31335 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:51.643193 31335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:51.643424 31335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:51.645478 31335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/data/instance:
uuid: "3b5d88532de34394bd163b70728172f9"
format_stamp: "Formatted at 2026-08-09 04:44:51 on dist-test-slave-vxj2"
server_key: "691f2ea5feb50c7d33ca0bf436a2197a"
server_key_iv: "a275d4d00f37e7180ffcb48d557819d2"
server_key_version: "encryptionkey@0"
I20260809 04:44:51.646039 31335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/wal/instance:
uuid: "3b5d88532de34394bd163b70728172f9"
format_stamp: "Formatted at 2026-08-09 04:44:51 on dist-test-slave-vxj2"
server_key: "691f2ea5feb50c7d33ca0bf436a2197a"
server_key_iv: "a275d4d00f37e7180ffcb48d557819d2"
server_key_version: "encryptionkey@0"
I20260809 04:44:51.649696 31335 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:51.652096 31351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:51.653344 31335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:51.653481 31335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/data,/tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/wal
uuid: "3b5d88532de34394bd163b70728172f9"
format_stamp: "Formatted at 2026-08-09 04:44:51 on dist-test-slave-vxj2"
server_key: "691f2ea5feb50c7d33ca0bf436a2197a"
server_key_iv: "a275d4d00f37e7180ffcb48d557819d2"
server_key_version: "encryptionkey@0"
I20260809 04:44:51.653590 31335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:51.678632 31335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:51.682008 31335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:51.682217 31335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:51.689359 31335 rpc_server.cc:307] RPC server started. Bound to: 127.5.222.252:42997
I20260809 04:44:51.689642 31411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.222.252:42997 every 8 connection(s)
I20260809 04:44:51.690418 31335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/data/info.pb
I20260809 04:44:51.693356 31412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:51.696940 31412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:51.699869 6011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu as pid 31335
I20260809 04:44:51.700039 6011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxJEoz3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250403492198-6011-0/minicluster-data/master-2/wal/instance
I20260809 04:44:51.700309 6011 external_mini_cluster.cc:1428] Setting key 4335048fd49f265719e021de1c883350
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250691, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.252@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:51.706909 31246 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } attempt: 3
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.252@KRBTEST.COM
I20260809 04:44:51.712112 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.698783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.5.222.254:33909)
0809 04:44:51.698919 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.698923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.699804 (+ 881us) server_negotiation.cc:407] Connection header received
0809 04:44:51.700848 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.700852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.700911 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.701014 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.702524 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.703041 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.704228 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.704461 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.704576 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.707213 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.707219 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.707226 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.707258 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.709409 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.710015 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.710019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.710021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.710109 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.710518 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.710522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.710524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.710778 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.710835 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.711692 (+ 857us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.711959 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250691, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.254@KRBTEST.COM for kudu/127.5.222.252@KRBTEST.COM
I20260809 04:44:51.712917 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.700959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34930 (local address 127.5.222.252:42997)
0809 04:44:51.701315 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.701329 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.701450 (+ 121us) server_negotiation.cc:407] Connection header received
0809 04:44:51.701537 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.701543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.701734 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.701904 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.702622 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.703463 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.704372 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.704745 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.704875 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.707752 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.707759 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.707770 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.707795 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.709785 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.710293 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.710298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.710299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.710392 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.710709 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.710712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.710713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.711143 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.711397 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.711573 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.711915 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":268,"thread_start_us":163,"threads_started":1}
I20260809 04:44:51.712921 31414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.699134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.254:33909 (local address 127.0.0.1:44568)
0809 04:44:51.699608 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.699650 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:51.700627 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:51.701232 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:51.701241 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:51.701749 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:51.702386 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.702400 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.703163 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.703167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.704100 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.704110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.704611 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.704616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.704784 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.705445 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:51.705469 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:51.707025 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:51.709518 (+ 2493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.709529 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.709538 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.709903 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.710218 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.710221 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.710223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.710402 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.710905 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:51.710909 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:51.711144 (+ 235us) client_negotiation.cc:769] Sending connection context
0809 04:44:51.711834 (+ 690us) client_negotiation.cc:240] Negotiation successful
0809 04:44:51.712019 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":5,"thread_start_us":104,"threads_started":1}
I20260809 04:44:51.714994 31412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:51.717459 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.707339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34934)
0809 04:44:51.707781 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.707795 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:51.707873 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:51.708431 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:51.708434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:51.708611 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:51.708733 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.708741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.709815 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.709821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.710592 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.710600 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.711125 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.711129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.711208 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.711644 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:51.711660 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:51.713187 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:51.715602 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.715605 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.715608 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.715861 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.716446 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.716449 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.716451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.716541 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.716909 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:51.716912 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:51.717012 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 04:44:51.717124 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 04:44:51.717285 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":5}
I20260809 04:44:51.718045 31417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.707524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34934 (local address 127.5.222.252:42997)
0809 04:44:51.708154 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.708181 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.708197 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:44:51.708236 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.708241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.708285 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.708380 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.708866 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.709680 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.710688 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.710997 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.711101 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.713339 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.713342 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.713343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.713363 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.715474 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.716266 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.716268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.716269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.716338 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.716637 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.716642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.716644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.716806 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.716913 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.717740 (+ 827us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.717849 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":102,"threads_started":1}
I20260809 04:44:51.720005 31329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } attempt: 3
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250691, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.252@KRBTEST.COM for kudu/127.5.222.253@KRBTEST.COM
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.254@KRBTEST.COM
I20260809 04:44:51.722295 31246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80: Bootstrap starting.
I20260809 04:44:51.725345 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.715589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50176 (local address 127.5.222.253:45811)
0809 04:44:51.715697 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.715701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.715766 (+ 65us) server_negotiation.cc:407] Connection header received
0809 04:44:51.715902 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.715909 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.715971 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.716052 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.716500 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.717311 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.718380 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.718717 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.718804 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.721741 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.721745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.721748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.721783 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.723413 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.723949 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.723952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.723954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.724036 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.724347 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.724351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.724352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.724552 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.724625 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.725006 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.725117 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":31}
I20260809 04:44:51.725553 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.715877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44584 (local address 127.5.222.254:33909)
0809 04:44:51.715968 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.715972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.716125 (+ 153us) server_negotiation.cc:407] Connection header received
0809 04:44:51.716210 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.716213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.716263 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.716338 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.717124 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.717822 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.718923 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.719191 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.719346 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.721875 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.721878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.721881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.721903 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.723633 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.724145 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.724148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.724150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.724237 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.724533 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.724536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.724538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.724728 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.724883 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.725274 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.725399 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 04:44:51.725809 31414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.715494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.253:45811 (local address 127.0.0.1:50176)
0809 04:44:51.715702 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.715716 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:51.715796 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:51.716059 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:51.716061 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:51.716242 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:51.716360 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.716366 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.717426 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.717429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.718226 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.718236 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.718843 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.718850 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.718948 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.719440 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:51.719454 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:51.721291 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:51.723540 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.723543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.723545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.723813 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.724158 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.724161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.724163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.724245 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.724699 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:51.724701 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:51.724806 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 04:44:51.725489 (+ 683us) client_negotiation.cc:240] Negotiation successful
0809 04:44:51.725651 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260809 04:44:51.726598 31246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:51.726660 31412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250691, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.253@KRBTEST.COM for kudu/127.5.222.252@KRBTEST.COM
I20260809 04:44:51.727602 31246 log.cc:824] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:51.729820 31246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80: No bootstrap required, opened a new log
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250691, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.222.252@KRBTEST.COM for kudu/127.5.222.252@KRBTEST.COM
I20260809 04:44:51.731930 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.720741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34942)
0809 04:44:51.720888 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.720903 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:51.720980 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:51.721657 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:51.721662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:51.721847 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:51.721955 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.721964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.723894 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.723897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.724701 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.724711 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.725391 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.725395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.725514 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.725972 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:51.725986 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:51.728302 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:51.730358 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.730361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.730363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.730607 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.730952 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.730955 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.730957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.731051 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.731469 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:51.731472 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:51.731557 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 04:44:51.731638 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 04:44:51.731759 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 04:44:51.732560 31417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.720939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.5.222.252:42997)
0809 04:44:51.721448 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.721452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.721462 (+ 10us) server_negotiation.cc:407] Connection header received
0809 04:44:51.721493 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.721496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.721531 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.721606 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.722988 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.723769 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.724872 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.725262 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.726639 (+ 1377us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.728440 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.728443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.728446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.728472 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.730238 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.730727 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.730732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.730734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.730822 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.731163 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.731166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.731169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.731358 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.732182 (+ 824us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.732259 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.732396 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447}
I20260809 04:44:51.734522 31246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } }
I20260809 04:44:51.734777 31246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d69441040a1479a9f96339d35726d80, State: Initialized, Role: FOLLOWER
I20260809 04:44:51.735131 31246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } }
Aug 09 04:44:51 dist-test-slave-vxj2 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.222.253@KRBTEST.COM
I20260809 04:44:51.735805 31329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c: Bootstrap starting.
I20260809 04:44:51.736223 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.727288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34948 (local address 127.5.222.252:42997)
0809 04:44:51.727466 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.727470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.727483 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:44:51.727516 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.727518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.727559 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.727647 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.728156 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.728887 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.729872 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.730194 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.730292 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.732330 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.732333 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.732336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.732357 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.733991 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.734553 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.734556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.734557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.734650 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.735088 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.735090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.735092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.735384 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.735532 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.735987 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.736096 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260809 04:44:51.736634 31246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:51.736681 31414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.727157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.222.252:42997 (local address 127.0.0.1:34948)
0809 04:44:51.727254 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.727267 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:51.727336 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:51.727681 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:51.727684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:51.727929 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:51.728030 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.728038 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.729005 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.729007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.729729 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.729740 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.730382 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.730385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.730451 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.730874 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:51.730888 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:51.732196 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:51.734113 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.734116 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.734118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.734448 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.734886 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:51.734889 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:51.734891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:51.734991 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:51.735567 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:51.735569 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:51.735650 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 04:44:51.735743 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 04:44:51.735865 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 04:44:51.739135 31329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:51.739749 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.727984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50192 (local address 127.5.222.253:45811)
0809 04:44:51.728081 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.728085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.728096 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:44:51.728127 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.728129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.728168 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.728240 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.729760 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.730506 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.731862 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.732190 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.732273 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.735991 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.735995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.735997 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.736021 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.737742 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.738225 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.738228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.738230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.738315 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.738676 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.738679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.738681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.738878 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.739045 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.739507 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.739614 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 04:44:51.740000 31412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9: Bootstrap starting.
I20260809 04:44:51.740111 31329 log.cc:824] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:51.742264 31329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c: No bootstrap required, opened a new log
I20260809 04:44:51.743471 6011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:51.743857 31412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:51.744735 31412 log.cc:824] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:51.744740 31216 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1d69441040a1479a9f96339d35726d80"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44584
I20260809 04:44:51.744956 31216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 04:44:51.745075 31216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:51.741891 31172 master_runner.cc:186] Error getting master registration for 127.5.222.253:45811: OK, instance_id { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" instance_seqno: 1786250691463856 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:44:51.745707 31172 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:44:51.746330 31216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } }
I20260809 04:44:51.746361 31329 raft_consensus.cc:348] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } }
I20260809 04:44:51.746551 31329 raft_consensus.cc:729] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f60f522ce9264954ae783c735ddbb83c, State: Initialized, Role: FOLLOWER
I20260809 04:44:51.746791 31329 consensus_queue.cc:260] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } }
I20260809 04:44:51.747253 31216 leader_election.cc:288] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [CANDIDATE]: Term 1 election: Requested vote from peers f60f522ce9264954ae783c735ddbb83c (127.5.222.253:45811), 3b5d88532de34394bd163b70728172f9 (127.5.222.252:42997)
I20260809 04:44:51.747642 31434 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } } }
I20260809 04:44:51.747850 31434 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:51.747962 31412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9: No bootstrap required, opened a new log
I20260809 04:44:51.748457 31329 sys_catalog.cc:564] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:51.748494 31300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d69441040a1479a9f96339d35726d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f60f522ce9264954ae783c735ddbb83c"
I20260809 04:44:51.748683 31300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:51.750114 31300 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d69441040a1479a9f96339d35726d80 in term 1.
I20260809 04:44:51.750550 31188 leader_election.cc:302] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d69441040a1479a9f96339d35726d80, f60f522ce9264954ae783c735ddbb83c; no voters:
W20260809 04:44:51.752087 31430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:51.752166 31430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:51.752404 31419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } } }
I20260809 04:44:51.752522 31419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:51.752604 31419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:51.752684 31419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [term 1 LEADER]: Becoming Leader. State: Replica: 1d69441040a1479a9f96339d35726d80, State: Running, Role: LEADER
I20260809 04:44:51.752404 31412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } }
I20260809 04:44:51.752878 31412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5d88532de34394bd163b70728172f9, State: Initialized, Role: FOLLOWER
I20260809 04:44:51.752996 31419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } }
I20260809 04:44:51.753304 31412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } }
I20260809 04:44:51.754534 31412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:51.754576 31381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d69441040a1479a9f96339d35726d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3b5d88532de34394bd163b70728172f9"
I20260809 04:44:51.756024 31381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:51.756160 31419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d69441040a1479a9f96339d35726d80. Latest consensus state: current_term: 1 leader_uuid: "1d69441040a1479a9f96339d35726d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } } }
I20260809 04:44:51.756263 31419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [sys.catalog]: This master's current role is: LEADER
W20260809 04:44:51.753093 31450 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:51.757282 31450 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:51.757889 31381 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d69441040a1479a9f96339d35726d80 in term 1.
I20260809 04:44:51.758922 31254 master_runner.cc:186] Error getting master registration for 127.5.222.252:42997: OK, instance_id { permanent_uuid: "3b5d88532de34394bd163b70728172f9" instance_seqno: 1786250691687329 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:44:51.762624 31254 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:44:51.762773 31453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } } }
I20260809 04:44:51.762941 31453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:51.766367 31458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:51.770980 31458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:51.771414 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.753455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50200 (local address 127.5.222.253:45811)
0809 04:44:51.753568 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.753572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.759414 (+ 5842us) server_negotiation.cc:407] Connection header received
0809 04:44:51.759593 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.759597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.759640 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.759690 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.761000 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.761779 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.763000 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.763417 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.763501 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.765523 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.765527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.765530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.765554 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.769126 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.769754 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.769758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.769759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.769848 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.770154 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.770158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.770159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.770400 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.770501 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.771129 (+ 628us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.771254 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:44:51.774880 31417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.754982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34954 (local address 127.5.222.252:42997)
0809 04:44:51.755079 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.755083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.762364 (+ 7281us) server_negotiation.cc:407] Connection header received
0809 04:44:51.762407 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.762410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.762452 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.762527 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.763640 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.764316 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.765526 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.765886 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.765965 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.767968 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.767971 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.767974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.767997 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.769780 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.770287 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.770290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.770292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.770370 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.770686 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.770689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.770691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.770990 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.773491 (+ 2501us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.774609 (+ 1118us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.774727 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 04:44:51.776746 31470 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:51.776849 31470 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:51.781750 31381 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [term 1 FOLLOWER]: Refusing update from remote peer 1d69441040a1479a9f96339d35726d80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:51.782534 31451 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 04:44:51.784298 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.755354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44600 (local address 127.5.222.254:33909)
0809 04:44:51.755459 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.755465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.766124 (+ 10659us) server_negotiation.cc:407] Connection header received
0809 04:44:51.769383 (+ 3259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.769387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.769436 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.769488 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.773818 (+ 4330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.774610 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.777298 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.777640 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.777746 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.779556 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:51.779560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:51.779563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:51.779591 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:51.781426 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.781947 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.781951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.781953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.782045 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:51.782391 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:51.782394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:51.782396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:51.782642 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:51.782782 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:51.783192 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 04:44:51.783361 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 04:44:51.788415 31453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d69441040a1479a9f96339d35726d80. Latest consensus state: current_term: 1 leader_uuid: "1d69441040a1479a9f96339d35726d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } } }
I20260809 04:44:51.788551 31453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:51.789692 31300 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [term 1 FOLLOWER]: Refusing update from remote peer 1d69441040a1479a9f96339d35726d80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:51.790324 31451 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:51.794492 31434 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d69441040a1479a9f96339d35726d80. Latest consensus state: current_term: 1 leader_uuid: "1d69441040a1479a9f96339d35726d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } } }
I20260809 04:44:51.794615 31434 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:51.797474 31419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d69441040a1479a9f96339d35726d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } } }
I20260809 04:44:51.797590 31419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:51.798130 31453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d69441040a1479a9f96339d35726d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } } }
I20260809 04:44:51.798238 31453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:51.798621 31472 mvcc.cc:205] Tried to move back new op lower bound from 7316482833528213504 to 7316482833423855616. Current Snapshot: MvccSnapshot[applied={T|T < 7316482833528213504}]
I20260809 04:44:51.804561 31451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d69441040a1479a9f96339d35726d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } } }
I20260809 04:44:51.804678 31451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:51.805236 31473 mvcc.cc:205] Tried to move back new op lower bound from 7316482833528213504 to 7316482833423855616. Current Snapshot: MvccSnapshot[applied={T|T < 7316482833528213504 or (T in {7316482833528213504})}]
I20260809 04:44:51.805766 31434 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d69441040a1479a9f96339d35726d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d69441040a1479a9f96339d35726d80" member_type: VOTER last_known_addr { host: "127.5.222.254" port: 33909 } } peers { permanent_uuid: "f60f522ce9264954ae783c735ddbb83c" member_type: VOTER last_known_addr { host: "127.5.222.253" port: 45811 } } peers { permanent_uuid: "3b5d88532de34394bd163b70728172f9" member_type: VOTER last_known_addr { host: "127.5.222.252" port: 42997 } } }
I20260809 04:44:51.805873 31434 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:51.810524 31458 catalog_manager.cc:1347] Generated new cluster ID: b309853a95c34b2db9bbe5d88a0200d4
I20260809 04:44:51.810585 31458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:51.825587 31458 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:51.826967 31458 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:51.844278 31458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d69441040a1479a9f96339d35726d80: Generated new TSK 0
I20260809 04:44:51.845199 31458 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:44:51.888392 31417 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:51.885786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34958 (local address 127.5.222.252:42997)
0809 04:44:51.885915 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.885920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.887072 (+ 1152us) server_negotiation.cc:407] Connection header received
0809 04:44:51.887480 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.887484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.887537 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.887592 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.888247 (+ 655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34958: BlockingRecv error: recv got EOF from 127.0.0.1:34958 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 04:44:51.889556 31333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:51.885219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50204 (local address 127.5.222.253:45811)
0809 04:44:51.885355 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.885360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.887275 (+ 1915us) server_negotiation.cc:407] Connection header received
0809 04:44:51.888757 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.888761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.888811 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.888862 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.889439 (+ 577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50204: BlockingRecv error: recv got EOF from 127.0.0.1:50204 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 04:44:51.891070 31251 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:51.886232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44612 (local address 127.5.222.254:33909)
0809 04:44:51.886349 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.886353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.890132 (+ 3779us) server_negotiation.cc:407] Connection header received
0809 04:44:51.890344 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.890350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.890400 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.890452 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.890945 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44612: BlockingRecv error: recv got EOF from 127.0.0.1:44612 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 04:44:51.895362 31333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:51.893876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50220 (local address 127.5.222.253:45811)
0809 04:44:51.893986 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.893990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.894479 (+ 489us) server_negotiation.cc:407] Connection header received
0809 04:44:51.894678 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.894681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.894749 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.894798 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.895272 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50220: BlockingRecv error: recv got EOF from 127.0.0.1:50220 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 04:44:51.900238 31333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:51.898858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.5.222.253:45811)
0809 04:44:51.898965 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.898969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.899409 (+ 440us) server_negotiation.cc:407] Connection header received
0809 04:44:51.899586 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.899589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.899634 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.899684 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.900152 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50226: BlockingRecv error: recv got EOF from 127.0.0.1:50226 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 04:44:51.905790 31333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:51.904325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50234 (local address 127.5.222.253:45811)
0809 04:44:51.904419 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.904423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.904877 (+ 454us) server_negotiation.cc:407] Connection header received
0809 04:44:51.905089 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.905092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.905139 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.905231 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.905703 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50234: BlockingRecv error: recv got EOF from 127.0.0.1:50234 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 04:44:51.935799 31333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:51.920189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50238 (local address 127.5.222.253:45811)
0809 04:44:51.920345 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.920349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.934506 (+ 14157us) server_negotiation.cc:407] Connection header received
0809 04:44:51.934703 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.934707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.934777 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.934885 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.935677 (+ 792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50238: BlockingRecv error: recv got EOF from 127.0.0.1:50238 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260809 04:44:51.958329 31333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:51.946290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50250 (local address 127.5.222.253:45811)
0809 04:44:51.946415 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.946420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.957123 (+ 10703us) server_negotiation.cc:407] Connection header received
0809 04:44:51.957376 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.957394 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.957463 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.957571 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.958208 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50250: BlockingRecv error: recv got EOF from 127.0.0.1:50250 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 04:44:51.986709 31333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:51.977101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.5.222.253:45811)
0809 04:44:51.977253 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.977258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.985619 (+ 8361us) server_negotiation.cc:407] Connection header received
0809 04:44:51.985799 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.985802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.985867 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.985969 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.986582 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50264: BlockingRecv error: recv got EOF from 127.0.0.1:50264 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 04:44:52.022485 31333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:52.021341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50276 (local address 127.5.222.253:45811)
0809 04:44:52.021521 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:52.021526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:52.021674 (+ 148us) server_negotiation.cc:407] Connection header received
0809 04:44:52.021779 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:52.021782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:52.021829 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:52.021902 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:52.022354 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50276: BlockingRecv error: recv got EOF from 127.0.0.1:50276 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260809 04:44:52.090256 31333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:52.088744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50286 (local address 127.5.222.253:45811)
0809 04:44:52.089004 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:52.089009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:52.089170 (+ 161us) server_negotiation.cc:407] Connection header received
0809 04:44:52.089354 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:52.089357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:52.089409 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:52.089494 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:52.090070 (+ 576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50286: BlockingRecv error: recv got EOF from 127.0.0.1:50286 (error 108)
Metrics: {"server-negotiator.queue_time_us":163}
W20260809 04:44:52.221441 31333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:52.220270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50302 (local address 127.5.222.253:45811)
0809 04:44:52.220437 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:52.220441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:52.220533 (+ 92us) server_negotiation.cc:407] Connection header received
0809 04:44:52.220650 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:52.220652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:52.220705 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:52.220857 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:52.221291 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50302: BlockingRecv error: recv got EOF from 127.0.0.1:50302 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 04:44:52.480825 31333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:52.479630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50318 (local address 127.5.222.253:45811)
0809 04:44:52.479781 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:52.479786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:52.479907 (+ 121us) server_negotiation.cc:407] Connection header received
0809 04:44:52.480078 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:52.480081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:52.480140 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:52.480231 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:52.480720 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50318: BlockingRecv error: recv got EOF from 127.0.0.1:50318 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:44:52.758682 31450 catalog_manager.cc:1259] Loaded cluster ID: b309853a95c34b2db9bbe5d88a0200d4
I20260809 04:44:52.758796 31450 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c: loading cluster ID for follower catalog manager: success
I20260809 04:44:52.760520 31450 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c: acquiring CA information for follower catalog manager: success
I20260809 04:44:52.761737 31450 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f60f522ce9264954ae783c735ddbb83c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:44:52.778388 31470 catalog_manager.cc:1259] Loaded cluster ID: b309853a95c34b2db9bbe5d88a0200d4
I20260809 04:44:52.778465 31470 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9: loading cluster ID for follower catalog manager: success
I20260809 04:44:52.780084 31470 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9: acquiring CA information for follower catalog manager: success
I20260809 04:44:52.781232 31470 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b5d88532de34394bd163b70728172f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:44:52.999169 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:52.995066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50322 (local address 127.5.222.253:45811)
0809 04:44:52.995317 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:52.995324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:52.995424 (+ 100us) server_negotiation.cc:407] Connection header received
0809 04:44:52.995601 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:52.995605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:52.995745 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:52.995871 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:52.996144 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.996678 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.997619 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.997855 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.997970 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:52.998298 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:52.998734 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:52.998877 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:52.998939 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 04:44:52.998998 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260809 04:44:53.005844 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:53.002148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34974 (local address 127.5.222.252:42997)
0809 04:44:53.002422 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:53.002426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:53.002439 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:44:53.002516 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:53.002519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:53.002627 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:53.002729 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:53.002896 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.003407 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.004246 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.004488 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.004588 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:53.004876 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:53.005377 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:53.005477 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:53.005546 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 04:44:53.005596 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":93,"threads_started":1}
I20260809 04:44:53.013528 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:53.008998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50324 (local address 127.5.222.253:45811)
0809 04:44:53.009128 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:53.009132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:53.009420 (+ 288us) server_negotiation.cc:407] Connection header received
0809 04:44:53.009617 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:53.009621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:53.009684 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:53.009737 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:53.010198 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.010969 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.012298 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.012593 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.012672 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:53.013001 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:53.013087 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:53.013198 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:53.013314 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 04:44:53.013381 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:44:53.013546 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:53.009027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34978 (local address 127.5.222.252:42997)
0809 04:44:53.009124 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:53.009129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:53.009568 (+ 439us) server_negotiation.cc:407] Connection header received
0809 04:44:53.009712 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:53.009714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:53.009826 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:53.009906 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:53.011083 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.011579 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.012445 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.012687 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.012787 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:53.013045 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:53.013109 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:53.013247 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:53.013399 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 04:44:53.013447 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 04:44:53.014732 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:53.008905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44628 (local address 127.5.222.254:33909)
0809 04:44:53.009223 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:53.009228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:53.009244 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:44:53.009439 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:53.009442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:53.009617 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:53.009769 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:53.010064 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.010920 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.012759 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.013119 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.013701 (+ 582us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:53.013868 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:53.014237 (+ 369us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:53.014349 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:53.014473 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 04:44:53.014542 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":106,"threads_started":1}
I20260809 04:44:53.021348 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 31172
I20260809 04:44:53.028630 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 31254
I20260809 04:44:53.036742 6011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxJEoz3/build/debug/bin/kudu with pid 31335
2026-08-09T04:44:53Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3256 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6423 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (289536 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST